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1.0  INTRODUCTION 


1.1  BACKGROUND 

Unexploded  Ordnance  (UXO)  detection  and  remediation  is  a  high  priority  triservice  requirement. 
As  the  Defense  Science  Board  recently  wrote:  “Today’s  UXO  cleanup  problem  is  massive  in 
scale  with  some  10  million  acres  of  land  involved.  Estimated  cleanup  costs  are  uncertain  but  are 
clearly  tens  of  billions  of  dollars.  This  cost  is  driven  by  the  digging  of  holes  in  which  no  UXOs 
are  present.  The  instruments  used  to  detect  UXOs  (generally  located  underground)  produce 
many  false  alarms,  -  i.e.,  detections  from  scrap  metal  or  other  foreign  or  natural  objects  -  ,  for 
every  detection  of  a  real  unexploded  munition  found.”  [1] 

There  is  general  agreement  that  the  best  solution  to  the  false  alarm  problem  involves  the  use  of 
EMI  sensors  which,  in  principle,  allow  the  extraction  of  target  shape  parameters  in  addition  to  a 
size  and  depth  estimate.  We,  and  others,  have  fielded  systems  with  either  time-domain  or 
frequency-domain  EMI  sensors  with  the  goal  of  extracting  reliable  target  shape  parameters  and, 
thus,  improving  the  discrimination  capability  of  our  surveys.  In  practice,  the  discrimination 
ability  of  these  sensors  has  been  limited  by  signal-to-noise  limitations.  Three  of  the  largest  noise 
terms  are  inherent  sensor  noise,  motion-induced  noise,  and  sensor  location  uncertainty. 

The  three  most  successful  demonstrations  of  EMI-based  discrimination  all  involved  cued 
detection  with  gridded  collection  of  EMI  data  [2-4].  The  success  of  the  gridded  data  collections 
was  due  to  the  combination  of  minimal  location  uncertainty,  no  motion-induced  noise,  and 
sufficient  SNR.  The  downside  of  the  implementations  previously  demonstrated  is  that  they  were 
relatively  slow  and  inefficient,  especially  on  a  large  site.  We  have  constructed  an  EMI  sensor 
array  that  combines  the  classification  ability  of  a  gridded  survey  with  the  coverage  efficiency  of 
a  vehicular  array.  By  coming  to  a  stop  over  each  target  to  be  investigated  we  are  able  to  obtain 
all  the  benefits  of  a  gridded  survey  (negligible  relative  sensor  location  uncertainty,  no  motion- 
induced  noise,  and  high  SNR)  while  moving  rapidly  to  the  next  target  with  no  set-up  required 
gives  us  the  coverage  efficiency  required  for  practical  success. 

1.2  OBJECTIVE  OF  THE  DEMONSTRATION 

The  objective  of  this  demonstration  was  to  validate  the  technology  through  a  blind  test  at  one  of 
the  Standardized  UXO  Test  Sites.  We  have  previously  done  a  shake-down  demonstration  of  the 
technology  at  our  Blossom  Point,  MD  field  site  but  a  blind  test  is  the  only  true  measure  of 
system  performance. 

1.3  REGULATORY  DRIVERS 

Stakeholder  acceptance  of  the  use  of  discrimination  techniques  on  real  sites  will  require 
demonstration  that  these  techniques  can  be  deployed  efficiently  and  with  high  probability  of 
discrimination.  The  first  step  in  this  process  is  to  demonstrate  acceptable  performance  on  a  test 
site  such  as  that  at  Aberdeen.  After  that  hurdle  has  been  passed,  successful  demonstration  at  a 
live  site  will  facilitate  regulatory  acceptance  of  the  methods. 
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2.0 


TECHNOLOGY 


2.1  TECHNOLOGY  DESCRIPTION 

2.1.1  EMI  Sensors 

The  EMI  sensor  used  in  the  Discrimination  Array  is  based  on  the  Navy-funded  Advanced 
Ordnance  Locator  (AOL),  developed  by  G  &  G  Sciences.  The  AOL  consists  of  three  transmit 
coils  arranged  in  a  1-m  cube;  we  have  adopted  the  transmit  (Tx)  and  receive  (Rx)  subsystems  of 
this  sensor  directly,  converted  to  a  5  x  5  array  of  35  cm  sensors,  and  made  minor  modifications 
to  the  control  and  data  acquisition  computer  to  make  it  compatible  with  our  deployment  scheme. 

A  photograph  of  an  individual  sensor  element  under  construction  is  shown  in  the  left  panel  of 
Figure  2-1.  The  transmit  coil  is  wound  around  the  outer  portion  of  the  form  and  is  35  cm  on  a 
side.  The  25-cm  receive  coil  is  wound  around  the  inner  part  of  the  form  which  is  re-inserted  into 
the  outer  portion.  An  assembled  sensor  with  the  top  and  bottom  caps  used  to  locate  the  sensor  in 
the  array  is  shown  in  the  right  panel  of  Figure  2-1. 


Figure  2-1  -  Constmction  details  of  an  individual  EMI  sensor  (left  panel)  and  the  assembled  sensor  with 
end  caps  attached  (right  panel). 

Decay  data  are  collected  with  a  500  kHz  sample  rate  until  25ms  after  turn  off  of  the  excitation 
pulse.  This  results  in  a  raw  decay  of  12,500  points;  too  many  to  be  practical.  These  raw  decay 
measurements  are  grouped  into  1 1 5  logarithmically-spaced  “gates”  whose  center  times  range 
from  42  ps  to  24.35  ms  with  5%  widths  and  are  saved  to  disk.  An  example  of  the  measured 
transmit  pulse,  raw  decay,  and  gated  decay  is  shown  in  Figure  2-2. 

The  individual  sensors  (consisting  of  transmit  electronics,  transmit  and  receive  coils,  pre-amp, 
and  digitizer)  were  characterized  at  G  &  G  Sciences  before  approval  was  given  for  construction 
of  the  array.  Examples  of  the  characterization  data  are  shown  in  Figure  2-3  and  Figure  2-4. 
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System  stability  is  shown  in  Figure  2-3  which  plots  the  normalized  (by  measured  transmit 
current)  response  of  a  2  in  steel  ball  at  25  cm  separation  from  the  sensor.  The  data  plotted  are 
decays  1,  1001,  2001,  and  3001  in  a  continuously-triggered  series  that  began  from  a  cold  start 
and  ran  for  150  minutes.  For  comparison  purposes,  the  expected  response  from  this  sphere  is 
plotted  in  black.  As  can  be  seen,  the  sensor  exhibits  excellent  stability  which  will  be  important 
for  the  cued  deployment  planned. 
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Figure  2-2  -  Measured  transmit  current  (upper  panel),  full  measured 
signal  decay  (middle  panel),  and  gated  decay  (lower  panel)  as  discussed 
in  the  text. 


The  second  important  characterization  test  is  sensor  response  linearity.  Since  we  plan  to  collect 
decay  data  to  late  times  and  over  several  orders  of  magnitude  in  amplitude,  the  linearity  of 
system  response  is  very  important.  To  characterize  this  property  of  the  sensor,  we  constructed  a 
series  of  copper  coils  with  nominal  decay  time  constants  of  2,  4,  and  6  ms.  The  response  of  the 
three  coils  is  shown  in  Figure  2-4  which  plots  the  measured  decay  on  a  semi-log  axes.  After  a 
transient  at  early  times,  the  decays  exhibit  clean  exponential  behavior  with  measured  decay  times 
of  1.8,  3.3,  and  5.8  ms.  Careful  calculation  of  the  expected  decay  times  at  the  temperature  at 
which  the  tests  were  conducted  results  in  expected  values  of  1.82,  3.26,  and  5.73  ms;  the 
measured  values  are  in  excellent  agreement  with  these. 
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Figure  2-3  -  Measured  response  from  a  2-in  steel  sphere  25  cm  from  the 
sensor.  Decays  1,  1001,  2001,  and  3001  from  a  series  that  started  from  a 
cold  start  are  plotted  along  with  the  expected  response  from  this  target. 


102 

>  10^ 

S)  10° 

(/) 

I  10-^ 

03 
CD 

10-2 

10-3 

0  5  10  15  20  25 

Time  (ms) 

Figure  2-4  -  Measured  response  from  three  calibration  coils  and  the 
background  response  between  measurements  plotted  on  a  semi-log  plot 
to  emphasize  the  exponential  nature  of  the  decay.  The  decay  time 
constants  extracted  from  the  measurements  are  listed  in  the  legend. 
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2.1.2  Sensor  Array 


The  twenty- five  individual  sensors  are  arranged  in  a  5  x  5  array  as  shown  in  Figure  2-5.  The 
center-to-center  distance  is  40  cm  yielding  a  2  m  x  2  m  array.  Also  shown  in  Figure  2-5  is  the 
position  of  the  three  GPS  antennae  that  are  used  to  determine  the  location  and  orientation  of  the 
array  for  each  cued  measurement.  A  picture  of  the  array  mounted  on  the  MTADS  EMI  sensor 
platform  is  shown  in  Figure  2-6. 


Figure  2-5  -  Sketch  of  the  EMI  sensor  array  showing  the  position  of  the 
25  sensors  and  the  three  GPS  antennae. 


Figure  2-6  -  Sensor  array  mounted  on  the  MTADS  EMI  sensor  platform. 
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After  assembly  of  the  array,  a  number  of  array  calibration  measurements  were  performed.  The 
first  task  was  to  ensure  that  each  of  the  individual  sensors  has  equivalent  response.  A  jig  was 
constructed  that  allows  us  to  mount  a  2-in  steel  sphere  30  cm  below  each  array  element  in  turn. 
Data  collected  using  this  jig  are  shown  in  Figure  2-7.  As  can  be  seen,  the  measured  decays  from 
each  of  the  sensors  plotted  are  indistinguishable. 


Figure  2-7  -  Comparison  of  the  response  of  the  array  members.  The 
measured  decay  from  a  2-in  steel  sphere  held  30  cm  below  each  sensor  in 
turn  is  plotted.  The  decays  are  indistinguishable. 

After  this,  the  assembled  array  was  used  to  measure  the  response  of  a  number  of  inert  ordnance 
items  and  stimulants  mounted  on  a  test  stand,  placed  in  our  test  pit,  or  buried  in  our  test  field. 
For  each  series  of  measurements  with  the  full  array,  we  cycle  through  the  sensors  transmitting 
from  each  in  turn.  After  each  excitation  pulse,  we  record  the  response  of  all  twenty-five  receive 
coils.  Thus,  there  are  625  (25  x  25)  individual  transmit/receive  pairs  recorded,  making  it  difficult 
to  present  a  full  measurement  in  any  coherent  way.  In  Figure  2-8,  we  plot  nine  of  the 
transmit/receive  pairs  resulting  from  excitation  of  a  40-mm  projectile  located  under  the  center  of 
the  array.  The  decays  plotted  correspond  to  the  signal  received  on  the  nine  central  sensors 
(reference  Figure  2-5  for  the  sensor  numbering)  when  that  sensor  transmits.  In  other  words,  the 
results  of  nine  individual  monostatic  measurements  are  presented. 

All  625  measurements  are  used  for  the  inversion  to  recover  target  parameters.  The  inversion 
results  for  the  decay  data  shown  in  Figure  2-8  are  shown  in  Figure  2-9.  As  we  expect  for  an 
object  with  axial  symmetry  such  as  a  40-mm  projectile,  we  recover  one  large  response 
coefficient  and  two  equal,  but  smaller  ones.  These  response  coefficients  will  be  the  basis  of  the 
discrimination  decisions  in  this  demonstration.  Derived  ps  for  “Cylinder  E”  (3"  x  12"  steel 
cylinder)  in  the  test  field  are  shown  for  comparison  in  Figure  2-10. 
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Figure  2-8  -  The  response  of  nine  of  the  individual  sensors  to  a  40-mm  projectile  located 
under  the  center  of  the  array. 
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Figure  2-9  -  Derived  response  coefficients  for  a  40-mm  projectile  using 
the  measurements  of  which  the  decays  shown  in  Figure  2-8  are  a  subset. 


Figure  2-10  -  Derived  response  coefficients  from  a  cued  measurement 
over  "Cylinder  E"  in  the  test  field. 

The  final  array  characterization  test  was  to  confirm  that  the  response  coefficients  we  recover  are 
invariant  to  object  position  and  orientation  under  the  array.  Figure  2-11  shows  the  derived  ps 
plotted  for  a  4.2-in  mortar  baseplate  after  measurements  at  three  position/orientation  pairs.  As 
can  be  seen,  the  inversion  results  are  robust  to  variation  in  the  object’s  position  and  orientation. 
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time  (ms) 

Figure  2-1 1  -  Three  sets  of  ps  derived  from  three  measurements  over  a 
4.2-in  mortar  baseplate  at  different  position/orientation  pairs. 

2.1.3  Application  of  the  Technology 

Application  of  this  technology  is  straightforward.  A  list  of  target  positions  is  developed  from  a 
survey  by  some  geophysieal  instrument;  in  the  case  of  this  demonstration,  the  MTADS 
magnetometer  array.  This  target  file,  containing  the  target  location  and  an  optional  flag  for 
additional  ‘stacking’  or  averaging,  is  transferred  to  the  system  eontrol  program  which  uses  the 
information  from  the  three  GPS  antennae  to  guide  the  operator  to  position  the  array  over  eaeh 
target  in  turn.  When  positioned  over  the  target,  the  data  aequisition  computer  steps  through  the 
array  sensors  sequentially,  just  as  in  the  eharaeterization  measurements  diseussed  in  the 
preeeding  seetion,  and  colleets  decays  from  all  twenty-five  receive  eoils  for  eaeh  exeitation. 
These  data  are  then  inverted  for  target  loeation  and  eharacteristics.  At  the  end  of  the  EMI  data 
collection,  a  few  seconds  of  platform  position  and  orientation  data  are  colleeted  to  be  used  to 
translate  the  inverted  target  position,  which  is,  of  course,  relative  to  the  array,  to  absolute 
position  and  orientation. 

In  the  final  version  of  this  teehnology,  the  inversion  will  be  performed  while  the  operator  is 
driving  the  array  to  the  next  target.  For  this  demonstration,  we  performed  the  inversions  off-line 
so  that  we  would  have  the  ability  to  intervene  in  the  automatic  process  as  required.  The  EMI  and 
position  data  were  transferred  to  the  analyst  several  times  eaeh  day  for  near  real-time  analysis  at 
the  demonstration  site. 

2.2  ADVANTAGES  AND  LIMITATIONS  OF  THE  TECHNOLOGY 

The  Discrimination  Array  is  designed  to  eombine  the  data  quality  advantages  of  a  gridded  survey 
with  the  eoverage  efficieneies  of  a  vehieular  system.  The  design  goal  of  this  system  is  to  eollect 
data  equal,  if  not  better,  in  quality  to  the  best  gridded  surveys  (the  relative  position  and 
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orientation  of  the  sensors  being  known  better  than  for  gridded  data)  while  proseeuting  many 
more  targets  each  field  day. 

There  are  obvious  limitations  to  the  use  of  this  technology.  The  array  is  a  2-m  square  so  fields 
where  the  vegetation  or  topography  interferes  with  passage  of  a  trailer  that  size  will  not  be 
amenable  to  the  use  of  the  present  array.  During  the  course  of  this,  and  subsequent, 
demonstrations  we  will  be  evaluating  the  utility  of  a  3  x  3  sub-array  of  sensors.  If  we  can  utilize 
the  sub-array,  we  can  increase  the  number  of  sites  for  which  this  technology  will  be  appropriate. 
The  other  serious  limitation  will  be  anomaly  density.  For  all  sensors,  there  is  a  limiting  anomaly 
density  above  which  the  response  of  individual  targets  cannot  be  separated.  We  have  chosen 
relatively  small  sensors  for  this  array  which  should  help  with  this  problem  but  we  cannot 
eliminate  it.  Based  on  experiments  at  our  test  pit  at  Blossom  Point,  the  results  of  this 
demonstration,  and  work  done  on  the  Camp  Sibert  data  sets,  anomaly  densities  of  300 
anomalies/acre  or  higher  would  limit  the  applicability  of  this  system  as  more  than  20%  of  the 
anomalies  would  have  another  anomaly  within  a  meter. 
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3.0  PERFORMANCE  OBJECTIVES 


The  performance  objectives  for  this  demonstration  are  summarized  in  Table  3-1.  Since  the 
Discrimination  Array  is  a  discrimination  technology,  the  performance  objectives  focus  on  the 
second  step  of  the  UXO  survey  problem;  we  assume  that  the  anomalies  from  all  targets  of 
interest  have  been  detected  and  included  on  the  target  list  that  we  worked  from. 

Table  3-1  -  Performance  Objectives  for  This  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Quantitative  Performance  Objectives 

Reduction  of  False 
Alarms 

Number  of  false 
alarms  eliminated  at 
demonstrator 
operating  point. 

•  Prioritized  dig  list 

•  Scoring  report  from 
APG 

Reduction  of  false 
alarms  by  >  50% 
with  95%  correct 
identification  of 
munitions 

Location  Accuracy 

Average  error  and 
standard  deviation  in 
both  axes  for 
interrogated  items 

•  Estimated  location 
from  analysis 

•  Scoring  report  from 
APG 

AN  and  AE  <  10  cm 
ctN  and  ctE  <  15  cm 

Production  Rate 

Number  of  targets 
interrogated  each  day 

•  Log  of  field  work 

75  targets  per  day 

Analysis  Time 

Average  time 
required  for  inversion 
and  classification 

•  Log  of  analysis  work 

15  min  per  target 

Qualitative  Performance  Objective 

Ease  of  Use 

•  Feedback  from 
operator  on  ease  of 
use 

Operator  comes  to 
work  smiling 

3.1  OBJECTIVE:  REDUCTION  OE  EALSE  ALARMS 

This  is  the  primary  measure  of  the  effectiveness  of  this  technology.  By  collecting  high-quality, 
precisely-located  data,  we  expect  to  be  able  to  discriminate  munitions  from  scrap  and  frag  with 
high  efficiency. 
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3.1.1  Metric 


At  a  seeded  test  site  such  as  the  APG  standardized  test  site,  the  metric  for  false  alarm  elimination 
is  straightforward.  We  prepared  a  ranked  dig  list  for  the  targets  we  interrogated  with  a  dig/no¬ 
dig  threshold  indicated  and  ATC  personnel  used  their  automated  scoring  algorithms  to  assess  our 
results. 

3.1.2  Data  Requirements 

The  identification  of  most  of  the  items  in  the  test  field  is  known  to  the  test  site  operators.  Our 
ranked  dig  list  is  the  input  for  this  standard  and  ATC’s  standard  scoring  is  the  output. 

3.1.3  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  more  than  50%  of  the  non-munitions  items  were 
labeled  as  no-dig  while  retaining  95%  of  the  munitions  items  on  the  dig  list. 

3.2  OBJECTIVE:  LOCATION  ACCURACY 

An  important  measure  of  how  efficiently  any  required  remediation  will  proceed  is  the  accuracy 
of  predicted  location  of  the  targets  marked  to  be  dug.  Large  location  errors  lead  to  confusion 
among  the  UXO  techs  assigned  to  the  remediation  costing  time  and  often  leading  to  removal  of  a 
small,  shallow  object  when  a  larger,  deeper  object  was  the  intended  target. 

3.2.1  Metric 

As  above,  the  metric  for  location  accuracy  is  straightforward  at  a  seeded  test  site  such  as  the 
APG  standardized  test  site.  We  provided  an  estimated  position  for  all  targets  we  interrogated 
and  ATC  personnel  used  their  automated  scoring  algorithms  to  assess  our  results. 

3.2.2  Data  Requirements 

The  location  of  most  of  the  items  in  the  test  field  is  known  to  the  test  site  operators.  Our  dig  list 
is  the  input  for  this  standard  and  ATC’s  standard  scoring  is  the  output. 

3.2.3  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  the  average  position  error  was  less  than  10  cm  in 
both  dimensions  (low  bias)  and  the  standard  deviation  of  each  dimension  was  less  than  15  cm 
(accurate  location). 

3.3  OBJECTIVE:  PRODUCTION  RATE 

Even  if  the  performance  of  the  technology  on  the  two  metrics  above  is  satisfactory,  there  is  an 
economic  metric  to  consider.  There  is  a  known  cost  of  remediating  a  suspected  munitions  item. 
If  the  cost  to  interrogate  a  target  is  greater  than  this  cost,  the  technology  will  be  useful  only  at 
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sites  with  special  conditions  or  target  values.  Note,  however,  that  in  its  ultimate  implementation 
this  technology  will  result  in  reacquisition,  cued  interrogation,  and  target  flagging  in  one  visit  to 
the  site. 

3.3.1  Metric 

The  number  of  targets  interrogated  per  day  is  the  metric  for  this  objective.  Combined  with  the 
daily  operating  cost  of  the  technology  this  gives  the  per-item  cost. 

3.3.2  Data  Requirements 

Survey  productivity  was  determined  from  a  review  of  the  ATC  demonstration  field  logs. 

3.3.3  Success  Criteria 

For  this  first  demonstration,  the  objective  will  be  considered  to  be  met  if  at  least  75  targets  were 
interrogated  each  survey  day. 

3.4  OBJECTIVE:  ANALYSIS  TIME 

The  ultimate  implementation  of  this  technology  will  involve  on-the-fly  analysis  and 
classification.  The  time  for  this  will  be  limited  to  the  driving  time  to  the  next  anomaly  on  the 
list.  We  will  track  the  near-real-time  analysis  time  in  this  demonstration. 

3.4.1  Metric 

The  time  required  for  inversion  and  classification  per  anomaly  is  the  metric  for  this  objective 

3.4.2  Data  Requirements 

Analysis  time  was  determined  from  a  review  of  the  data  analysis  logs. 

3.4.3  Success  Criteria 

For  this  first  demonstration,  the  objective  will  be  considered  to  be  met  if  the  average  inversion 
and  classification  time  was  less  than  1 5  min. 

3.5  OBJECTIVE:  EASE  OF  USE 

This  qualitative  objective  is  intended  as  a  measure  of  the  long-term  usability  of  the  technology. 
If  the  operator  does  not  report  that  the  technology  is  easy  to  use,  shortcuts  that  can  compromise 
the  efficiency  of  the  technology  will  begin  to  creep  into  daily  operations. 

3.5.1  Data  Requirements 

This  objective  was  evaluated  based  on  operator  feedback. 
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4.0  SITE  DESCRIPTION 


This  demonstration  was  conducted  at  the  APG  Standardized  UXO  Technology  Demonstration 
Site  located  at  the  Aberdeen  Proving  Ground,  MD  during  the  period  of  May  through  June,  2008. 

4.1  SITE  SELECTION 

This  was  our  first  field  demonstration  of  this  combination  of  EMI  sensors  and  survey  mode.  As 
such,  the  demonstration  was  conducted  on  the  Standardized  UXO  Test  Site  at  APG.  The  APG 
site  is  located  close  to  our  base  of  operations  in  southern  Maryland  and  therefore  minimizes  the 
logistics  costs  of  the  deployment.  Use  of  this  site  allows  us  to  receive  validation  results  from 
near-real-world  conditions  without  incurring  the  logistics  and  intrusive  investigation  expenses 
that  would  be  required  for  a  demonstration  at  a  live  site. 

4.2  SITE  HISTORY 

The  Standardized  UXO  Technology  Demonstration  Site  is  adjacent  to  the  Trench  Warfare 
facility  at  the  Aberdeen  Proving  Ground.  The  specific  area  was  used  for  a  variety  of  ordnance 
tests  over  the  years.  Initial  magnetometer  and  EMI  surveys  conducted  by  the  MTADS  team 
performed  after  a  “mag  and  flag”  survey  of  the  same  area  identified  over  a  thousand  remaining 
anomalies.  These  data  were  used  for  a  final  clean  up  of  the  site  prior  to  the  emplacement  of  the 
original  test  items.  Prior  to  the  two  subsequent  reconfiguration  events,  unexplained  anomalies 
identified  by  demonstrators  using  the  site  were  also  investigated  and  removed. 

4.3  SITE  TOPOGRAPHY  AND  GEOLOGY 

According  to  the  soils  survey  conducted  for  the  entire  area  of  APG  in  1998,  the  test  site  consists 
primarily  of  Elkton  Series  type  soil  [5].  The  Elkton  Series  consist  of  very  deep,  slowly 
permeable,  poorly  drained  soils.  These  soils  formed  in  silty  aeolin  sediments  and  the  underlying 
loamy  alluvial  and  marine  sediments.  They  are  on  upland  and  lowland  flats  and  in  depressions 
of  the  Mid- Atlantic  Coastal  Plain.  Slopes  range  from  0  to  2  percent. 

Overall,  the  demonstration  site  is  relatively  flat  and  level.  There  are  some  low-lying  areas  in  the 
northwest  portion  of  the  site  that  tend  to  have  standing  water  during  the  wet  periods  of  the  year. 
The  current  sensor  system  is  not  sufficiently  weatherproofed  to  operate  through  standing  water. 
However,  during  the  most  recent  reconfiguration,  the  areas  most  prone  to  being  underwater  were 
excluded  from  the  survey  scenarios.  Anomalies  that  were  located  underwater  or  nearby  to  water 
at  the  time  of  survey  were  deferred  until  the  end  of  the  survey  and  were  interrogated  by  carefully, 
if  less  efficiently,  maneuvering  the  array  into  position. 

4.4  MUNITIONS  CONTAMINATION 

The  area  currently  occupied  by  the  Standardized  site  has  seen  an  extensive  history  of  munitions 
use.  As  an  example,  in  2003  we  conducted  a  magnetometer  survey  of  a  previously  unremediated 
area  directly  adjacent  to  the  Standardized  site  [6].  In  a  survey  area  of  approximately  1  hectare. 
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we  identified  2479  anomalies,  of  which  1921  were  amenable  to  a  model  fit  using  our  standard 
analysis.  Historical  records  provided  by  ATC  and  previous  remediation  results  indicated  that  the 
likely  munitions  of  interest  for  this  site  were: 

•  Grenades,  Mkl,  Mkll,  and  French  VB  Rifle  w/o  chute 

•  Grenades,  French  VB  Rifle  w/  chute 

•  60mm  mortars  (including  2”  Smoke) 

•  3”  Stokes  (Smoke  and  HE) 

•  105  mm  projectiles 

•  155  mm  proj  ectiles 

4.5  SITE  GEODETIC  CONTROL  INFORMATION 

There  are  two  first-order  points  on  the  site  for  use  as  GPS  base  station  points.  Their  reported 
coordinates  are  listed  in  Table  4-1.  The  horizontal  datum  for  all  values  is  NAD83.  The  vertical 
control  is  referenced  to  the  NAVD88  datum  and  the  Geoid03  geoid.  Point  477  was  used  as  the 
GPS  base  station  point  for  the  entirety  of  this  demonstration. 

Table  4-1  -  Geodetic  Control  at  the  APG  Demonstration  Site 


ID 

Latitude 

Longitude 

Elevation 

Northing 

Easting 

HAE 

477 

39^^28’  18.63880”  N 

76°  07’  47.71815”W 

10.669  m 

4,369,749.013 

402,810.038 

-22.545 

478 

39^^28’  04.24219”  N 

76°  07’  48.50439”W 

11.747  m 

4,369,305.416 

402,785.686 

-21.473 

4.6  SITE  CONFIGURATION 

Figure  4-1  is  a  map  of  the  Standardized  UXO  Technology  Demonstration  Site  at  APG.  The 
Calibration  and  Blind  Grids  are  shown  along  with  the  various  Open  Field  Areas. 
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Figure  4-1  -  Map  of  the  reconfigured  APG  Standardized  UXO  Test  Site. 
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5.0  TEST  DESIGN 


5.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

The  demonstration  was  designed  to  be  exeeuted  in  two  stages.  The  first  stage  was  a  standard 
MTADS  magnetometer  survey  of  the  Calibration,  Blind  Grid,  and  the  Indireet  Fire  Area  of  the 
Standardized  Site.  An  experienced  data  analyst  manually  inspected  the  Blind  Grid  data  and 
made  a  determination  for  each  cell  as  to  whether  or  not  an  anomaly  was  present.  Anomaly 
locations  were  identified  from  the  magnetometer  data  from  the  Indirect  Fire  Area  in  a 
combination  automated  /  manual  method  as  described  in  Sections  5.5.9  and  6.2.  A  data  segment 
around  each  anomaly  center  was  extracted  and  analyzed  using  the  UX- Analyze  subsystem  of  the 
Oasis  montaj  software  package  as  described  in  Section  6.2  to  fit  the  data  to  a  dipole  model  and 
extract  the  associated  fit  parameters  (position,  depth,  equivalent  size).  These  fit  results 
constituted  the  source  anomaly  list  for  the  second  stage  of  the  demonstration. 

In  practice,  the  majority  of  the  site  was  surveyed  as  a  whole  with  the  magnetometer  system  and 
the  resulting  data  were  analyzed  in  parallel  to  the  above  effort.  The  results  were  provided  to  the 
Program  Office  in  support  of  the  recent  reconfiguration  effort.  Anomalies  were  detected  in  a 
manner  similar  to  that  used  for  MTADS  data  sets  collected  as  part  of  the  ESTCP  UXO 
Discrimination  Study  at  the  former  Camp  Sibert  [7]. 

This  method  relies  on  the  establishment  of  an  anomaly  detection  threshold.  At  the  former  Camp 
Sibert  demonstration  site,  a  single  munitions  type  was  present.  Pit  measurements  at  various 
depths  and  orientations  of  an  example  article  were  made  and  bounding  response  curves 
generated  for  the  4.2-in  mortar,  the  munitions  of  interest.  The  anomaly  detection  threshold  was 
then  set  based  on  the  least-favorably  predicted  response  at  the  USACoE  standard  1  lx  depth.  For 
our  recent  GEMTADS  demonstration  at  F.E.  Warren  AFB,  two  primary  munitions  types  were 
present,  37  and  75mm  projectiles.  In  this  case,  response  curves  were  generated  for  each 
munitions  type  and  the  smaller  of  the  two  indicated  detection  thresholds  was  selected  as  the 
overall  threshold.  In  the  reconfigured  Standardized  Site,  several  areas  have  been  established 
with  different  mixes  of  emplaced  munitions.  Individual  anomaly  detection  thresholds  were 
established  for  each  area  based  on  an  abbreviated  set  of  pit  measurements  made  at  Blossom  Point 
for  each  of  the  emplaced  items.  For  each  area,  the  smallest  appropriate  least-favorable  response 
will  be  used  to  determine  the  threshold.  A  safety  factor  of  25%  was  also  applied. 

As  was  shown  during  the  ESTCP  Project  200413  “Gold  Standard”  surveys,  the  dynamic 
background  signal  level  for  the  Standardized  Test  Site  at  APG  was  at  least  2.5  nT  for  the 
magnetometer  system.  The  dynamic  background  level  at  the  reconfigured  site  was  characterized 
prior  to  anomaly  selection. 

Individual  extracted  anomalies  were  analyzed  using  UX-Analyze.  We  were  able  to  survey  a  large 
majority  of  the  entire  site  excluding  the  Wooded  and  Mogul  Areas.  At  the  time  of  the 
demonstration,  the  Challenge  Area  was  unpopulated  and  was  also  not  surveyed. 
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The  second  stage  of  the  demonstration  was  a  survey  of  the  Calibration,  Blind  Grid,  and  the 
Indirect  Fire  Area  using  the  MTADS  Discrimination  Array  developed  as  part  of  ESTCP  Project 
MM-0601.  The  array  was  positioned  roughly  over  the  center  of  each  anomaly  on  the  source 
anomaly  list  and  a  data  set  collected.  Each  data  set  was  then  inverted  using  the  data  analysis 
methodology  discussed  in  Section  6.0,  estimated  target  parameters  determined,  and  ultimately  a 
classification  made  for  each  cell  in  the  Blind  Grid  determined  to  contain  an  anomaly  and  each 
anomaly  placed  on  the  Indirect  Fire  Area  anomaly  list.  The  results  were  then  submitted  to  the 
Aberdeen  Test  Center  for  performance  assessment. 


The  schedule  of  field  testing  activities  is  provided  in  Figure  5-1  as  a  Gantt  chart. 


Activity  Name 

May 2008 

Jun  2008 

4 

11 

18 

25 

1 

8 

15 

22 

1 

APG  Demonstration 

2 

Magnetometer  Data  Collection 

1 

3 

Discrimination  Data  Collection 

4 

11 

18 

25 

1 

8 

15 

22 

Figure  5-1  -  Schedule  of  Field  Testing  Activities. 

5.2  SITE  PREPARATION 

The  Standardized  UXO  Test  Sites  are  configured  with  clearly-marked  calibration,  blind 
validation,  and  open  field  scenarios.  Several  GPS  control  points  are  provided  at  each  site.  Basic 
facilities  such  as  portable  toilets  and  field  buildings  are  also  provided.  Secure  storage  for  larger 
vehicles  and  sensor  arrays  is  limited  at  the  APG  Test  Site.  A  40-foot  shipping  container  was 
mobilized  to  the  site  for  the  duration  of  the  Discrimination  Array  portion  of  the  demonstration  to 
provide  convenient,  secure  storage  for  the  MTADS  tow  vehicle  and  the  sensor  trailer.  The 
container  was  removed  at  the  end  of  the  demonstration. 

5.3  SYSTEMS  SPECIFICATION 

This  demonstration  was  conducted  using  the  NRL  MTADS  tow  vehicle  and  subsystems.  The 
tow  vehicle  and  each  subsystem  are  described  further  in  the  following  sections. 

5.3.1  MTADS  Tow  Vehicle 

The  MTADS  has  been  developed  with  support  from  ESTCP.  The  MTADS  hardware  consists  of 
a  low-magnetic-signature  vehicle  that  is  used  to  tow  the  different  sensor  arrays  over  large  areas 
(10  -  25  acres  /  day)  to  detect  buried  UXO.  The  MTADS  tow  vehicle  and  magnetometer  array 
are  shown  in  Figure  5-2. 
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Figure  5-2  -  MTADS  tow  vehicle  and  magnetometer  array. 

5.3.2  RTK  GPS  System 

Positioning  is  provided  using  cm-level  Real  Time  Kinematic  (RTK)  Global  Positioning  System 
(GPS)  receivers.  To  achieve  cm-level  precision,  a  fixed  reference  base  station  is  placed  on  an 
established  first-order  survey  control  point  near  the  survey  area.  The  base  station  transmits 
corrections  to  the  GPS  rover  at  1  Hz  via  a  radio  link  (450  MHz).  For  the  magnetometer  array,  a 
single  GPS  antenna  placed  directly  above  the  center  of  the  sensor  array  is  used  to  measure  the 
sensor  positions  in  real-time  (5  Hz)  using  a  vendor-specific  NMEA-0183  message  format 
(PTNL,GGK  or  GGK). 

The  Discrimination  Array  is  located  in  three-dimensional  space  using  a  three-receiver  RTK  GPS 
system  shown  schematically  in  Figure  2-5  [8].  The  three-receiver  configuration  extends  the 
concept  of  RTK  operations  from  that  of  a  fixed  base  station  and  a  moving  rover  to  moving  base 
stations  and  moving  rovers.  The  lead  GPS  antenna  (and  receiver,  Main)  receives  corrections 
from  the  fixed  base  station  at  1  Hz  in  the  same  manner  as  for  the  magnetometer  MTADS.  This 
corrected  position  is  reported  at  10-20  Hz.  The  Main  receiver  also  operates  as  a  ‘moving  base,’ 
transmitting  corrections  (by  serial  cable)  to  the  next  GPS  receiver  (AVRl)  which  uses  the 
corrections  to  operate  in  RTK  mode. 

A  vector  (AVRl,  heading  (yaw),  angle  (pitch),  and  range)  between  the  two  antennae  is  reported 
at  10  Hz  using  a  vendor-specific  NMEA-0183  message  format  (PTNL,AVR  or  AVR).  AVRl 
also  provides  ‘moving  base’  corrections  to  the  third  GPS  antenna  (AVR2)  and  a  second  vector 
(AVR2)  is  reported  at  10  Hz.  All  GPS  measurements  are  recorded  at  full  RTK  precision,  ~2-5 
cm.  For  survey-mode  arrays,  all  sensor  readings  are  typically  referenced  to  the  GPS  1-PPS  pulse 
output  to  fully  take  advantage  of  the  precision  of  the  GPS  measurements.  In  this  case  of  a  cued 
survey,  it  is  not  necessary  to  address  these  timing  issues.  For  the  cued-mode  survey,  the  GPS 
position  is  averaged  for  2  seconds  as  part  of  the  data  acquisition  cycle.  The  averaged  position 
and  orientation  information  are  then  recorded  to  the  position  (.gps,  ASCII  format)  data  file.  The 
details  of  the  file  format  are  provided  in  Appendix  B. 
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5.3.3  Magnetometer  Array 


The  MTADS  magnetometer  array  is  a  linear  array  of  eight  Cs-vapor  magnetometer  sensors 
(Geometries,  Inc.,  G-822ROV/A).  The  sensors  are  sampled  at  50  Hz  and  typical  surveys  are 
conducted  at  3  m/s.  This  results  in  a  sampling  density  of  ~6  cm  down  track  with  a  cross-track 
sensor  spacing  of  25  cm.  The  sensors  are  nominally  mounted  30  cm  above  the  ground.  The 
sensor  boom  is  designed  to  pivot  up  to  protect  the  sensors  from  damage  due  to  impact  with 
obstructions.  This  degree  of  freedom  allows  some  variation  in  sensor  height  due  to  surface 
roughness.  Each  magnetometer  measures  the  local  magnetic  field  of  the  earth  at  the  sensor. 

A  single  GPS  antenna  placed  directly  above  the  center  of  the  sensor  array  is  used  to  measure  the 
sensor  positions  in  real-time  (5  Hz).  All  navigation  and  sensor  data  are  time-stamped  with 
Universal  Coordinated  Time  (UTC)  derived  from  the  satellite  clocks  and  recorded  by  the  data 
acquisition  computer  (DAQ)  in  the  tow  vehicle.  The  DAQ  runs  the  MagLogNT  software 
package  (v2.921b.  Geometries,  Inc.)  and  the  data  streams  from  each  device  are  recorded  in 
separate  files  with  a  common  root  filename. 

5.3.4  Time-Domain  Electromagnetic  Sensor 

The  Discrimination  Array  is  a  5  x  5  square  array  of  individual  sensors.  Each  sensor  has 
dimensions  of  40  cm  x  40  cm,  for  an  array  of  2  m  x  2  m  overall  dimensions.  The  rationale  of 
this  array  design  is  discussed  in  Reference  9.  The  result  is  a  cross-track  and  down-track 
separation  of  40  cm.  Sensor  numbering  is  indicated  in  Figure  2-5.  The  transmitter  electronics 
and  the  data  acquisition  computer  are  mounted  in  the  tow  vehicle.  Custom  software  written  by 
NRL  provides  both  navigation  to  the  individual  anomalies  and  data  acquisition  functionality. 
After  the  array  is  positioned  roughly  centered  over  the  center  of  the  anomaly,  the  data  acquisition 
cycle  is  initiated.  Each  transmitter  is  fired  in  a  sequence  winding  outward  from  the  center 
position  (12)  in  a  clockwise  direction.  The  received  signal  is  recorded  for  all  25  Rx  coils  for 
each  transmit  cycle.  The  transmit  pulse  waveform  duration  is  2.7s  (0.9s  block  time,  9  repeats 
within  a  block,  3  blocks  stacked,  with  a  50%  duty  cycle).  While  it  is  possible  to  record  the  entire 
decay  transient  at  500  MHz,  we  have  found  that  binning  the  data  into  115  time  gates  simplifies 
the  analysis  and  provides  additional  signal  averaging  without  significant  loss  of  temporal 
resolution  in  the  transient  decays  [10].  The  data  are  recorded  in  a  binary  format  as  a  single  file 
with  25  data  points  (one  data  point  per  Tx  cycle).  The  filename  corresponds  to  the  anomaly 
under  investigation. 

5.4  CALIBRATION  ACTIVITIES 

For  the  magnetometer  array  data  collection,  no  specific  calibration  activities  beyond  the  standard 
operational  checks'  were  conducted  as  this  is  a  mature,  well-understood  technology. 
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See  Reference  7  for  a  discussion  of  our  standard  startup  operational  check  procedures. 
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For  the  Discrimination  Array,  a  significant  amount  of  data  has  been  collected  with  the  system  as 
configured  at  our  Blossom  Point  facility,  both  on  a  test  stand  and  in  the  towed  configuration  on 
our  test  field  [11].  These  data  and  the  corresponding  fit  parameters  provide  us  with  a  set  of 
reference  parameters  including  those  of  clear  background  (i.e.  no  anomaly  present). 

Daily  calibration  efforts  consist  of  collecting  background  (no  anomaly)  data  sets  at  the  beginning 
and  end  of  each  survey  day  and  periodically  throughout  the  day  at  quiet  spots  as  identified  from 
the  magnetometer  survey  to  determine  the  system  noise  floor.  A  canonical  reference  object,  a  4” 
A1  sphere,  was  placed  in  the  test  pit  located  near  the  Calibration  Area  and  measured  each  day  to 
monitor  the  daily  variation  in  the  system  response.  These  two  types  of  measurements  constituted 
the  daily  calibration  activities. 

The  sphere  measurements  are  background  subtracted  and  inverted  to  obtain  the  target 
polarizabilities.  These  data  allow  us  to  calibrate  our  sensor  response  in  two  separate  ways.  First, 
by  comparing  the  polarizabilities  derived  from  each  separate  measurement,  we  can  monitor  the 
daily  variation  in  the  system  response.  In  Figure  5-3,  we  plot  the  average  polarizability  for  all 
sphere  measurements  obtained  during  our  entire  run  as  a  function  of  decay  time.  Variations  of 
less  than  10%  of  the  reference  values  were  observed  over  the  course  of  eight  days. 


Time(ms) 

Figure  5-3  -  Average  polarizability  for  all  measurements  at  APG  of  the  4”  Aluminum  sphere  as  a 
function  of  decay  time 

Second,  we  compared  our  derived  polarizabilities  directly  with  the  predicted  decay  of  the  sphere. 
The  theoretical  decay  depends  on  the  target’s  size  and  conductivity.  We  have  determined  this 
latter  value  for  the  sphere  by  careful  measurements  with  a  host  of  sensors  over  the  years.  The 
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theoretical  decay  is  then  calculated  and  the  derived  polarizabilities  are  scaled  in  amplitude  to 
produce  the  best  match.  Figure  5-4  shows  the  result  for  one  measurement.  All  daily  calibration 
data  sets  are  included  on  the  accompanying  DVD. 
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Figure  5-4  -  Comparison  of  measured  (blue)  versus  theoretical  (red)  polarizability  as  a  function 
of  time  for  the  4”  Aluminum  sphere 

The  Calibration  Grid  at  APG  was  the  first  area  surveyed  and  the  results  were  compared  with  our 
reference  library  of  parameters  prior  to  completing  the  data  analysis  and  classification  of  the 
results  from  the  Blind  Grid  and  Indirect  Fire  Areas.  Prior  to  our  APG  demonstration,  we  had 
established  a  library  of  target  signatures  based  on  test  stand  measurements.  This  library  was 
further  expanded  based  on  sample  targets  measured  in  the  pit  at  APG.  Of  the  58  ordnance  targets 
present  in  the  Calibration  Grid,  we  were  able  to  correctly  identify  53  by  matching  the  inverted 
signatures  to  our  library.  Most  of  the  remaining  5  objects  suffered  from  low  signal-to-noise  or 
overlap  issues  with  nearby  targets.  In  Figure  5-5  through  Figure  5-7  we  show  the  match  between 
Calibration  Grid  cells  HI,  16,  and  K3,  and  our  library  60mm  and  81mm  Mortar  and  105mm 
Projectile,  respectively. 
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Time  (ms) 

Figure  5-5  -  Comparison  of  measured  (blue)  versus  library  (red)  polarizability  as  a  function  of 
time  for  Calibration  Area  item  HI,  60mm  Mortar 


Time  (ms) 

Figure  5-6  -  Comparison  of  measured  (blue)  versus  library  (red)  polarizability  as  a  function  of 
time  for  Calibration  Area  item  16,  81mm  Mortar 
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Time  (ms) 

Figure  5-7  -  Comparison  of  measured  (blue)  versus  library  (red)  polarizability  as  a  function  of 
time  for  Calibration  Area  item  K3,  105mm  Projectile 

As  this  was  the  first  demonstration  of  the  Discrimination  Array,  the  effect  of  background 
variation  from  point  to  point  and  over  time  was  not  fully  understood  and  remains  an  object  of 
study.  As  such,  tfequent  backgrounds  were  taken  over  the  course  of  each  field  day  for 
immediate  use  for  background  subtraction  and  future  analysis  regarding  issues  of  background 
variability  and  the  impact  on  fit  parameter  extraction.  Nine  background  locations  were  identified 
from  the  magnetometer  data  as  being  ‘quiet’  or  having  a  very  low  background  response  level. 
The  selected  locations  were  positioned  throughout  the  site  to  minimize  the  drive  time  to  a 
background  point  from  any  position  on  the  field.  The  locations  are  shown  in  Figure  5-8  and 
listed  in  Table  5-1. 
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4369350 4369400 4369450 4369500 4369550 4369600 4369650 4369700 4369750 


402650  402700  402750  402800  402850  402900  402950 


Figure  5-8  -  Discrimination  Array  background  measurement  locations.  The  boundaries  of  the 
ATC  test  field  areas  are  shown  for  reference. 
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4369750 4369700 4369650 4369600 4369550 4369500 4369450 4369400 4369350 


Table  5-1  -  Discrimination  Array  background  measurement  locations 


Fiducial 

Easting  (m) 

Northing  (m) 

Service  Area 

0 

402,812.00 

4,369,622.00 

Calibration  /  Blind  Grid 

1 

402,823.50 

4,369,521.00 

Challenge 

2 

402,807.50 

4,369,737.00 

Direct  Fire 

3 

402,746.40 

4,369,538.00 

IDF 

4 

402,735.90 

4,369,538.00 

IDF 

5 

402,725.00 

4,369,453.00 

IDF 

6 

402,758.10 

4,369,481.00 

IDF 

7 

402,805.50 

4,369,486.00 

IDF 

8 

402,687.00 

4,369,483.00 

IDF 

9 

402,769.40 

4,369,400.00 

IDF 

5.5  DATA  COLLECTION  PROCEDURES 

5.5.1  Scale  of  Demonstration 

The  demonstration  was  conducted  at  the  APG  Standardized  UXO  Test  Site.  A  magnetometer 
survey  was  conducted  on  the  Calibration  and  Blind  Grids,  as  well  as  the  Indirect  Fire  Area 
(approximately  4.3  acres)  on  May  7,  2008.  The  Discrimination  Array  surveyed  the  Calibration 
and  Blind  Grids.  The  array  was  also  deployed  to  approximately  700  anomalies  in  the  Indirect 
Fire  Area  that  were  detected  from  the  magnetometer  data  set.  The  Discrimination  Array  portion 
of  the  demonstration  occurred  from  June  16-23,  2008. 

5.5.2  Sample  Density 

Magnetometer  data  were  collected  with  nominal  down-track  spacing  of  6  cm  and  cross  track 
spacing  of  25  cm.  EMI  data  spacing  is  fixed  at  40  cm  in  both  directions  by  the  array  design. 

5.5.3  Quality  Checks 

Preventative  maintenance  inspections  were  conducted  continually  by  all  team  members  during 
data  collection,  focusing  particularly  on  the  tow  vehicle  and  sensor  trailer.  Deficiencies  were 
addressed  according  to  the  severity  of  the  deficiency.  Parts,  tools,  and  materials  for  many 
maintenance  scenarios  are  available  in  the  system  spares  inventory,  a  fraction  of  which  was  on 
site.  The  remainder  was  available  at  our  base  of  operations  at  Blossom  Point,  MD. 

For  the  magnetometer  array,  the  following  data  quality  checks  and  procedures  were  used  to 
insure  a  quality  data  product.  MTADS  magnetometer  survey  raw  data  generally  falls  into  two 
categories,  location  and  magnetometer  sensor  measurements.  The  data  set  is  comprised  of  ten 
separate  files,  each  containing  the  data  from  a  single  system  device.  Each  device  has  a  unique 
data  rate.  A  software  package  written  by  NRL  examines  each  file  and  compares  the  number  of 
entries  to  the  product  (total  survey  time  *  data  rate).  Any  discrepancies  are  flagged  for  the  data 
analyst  to  address. 
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For  magnetometer  sensor  data,  operational  values  are  typieally  on  the  order  of  50,000  nT  and 
have  noise  levels  of  -0.5  nT  peak-to-peak  (PP)  static  and  3-5  nT  PP  in  motion.  Sensor  “drop¬ 
outs”  can  occur  if  the  sensor  is  tilted  out  of  the  operation  zone  with  respect  to  the  earth’s 
magnetic  field.  If  a  sensor  cable  is  severed  or  damaged  while  in  motion,  the  sensor  output  value 
will  drop  below  20,000  nT  and/or  become  very  noisy  (1,000’s  of  nT  PP).  All  magnetometer 
sensor  channels  (8  total)  were  examined  in  each  survey  file  set  for  these  conditions  and  any  data 
which  are  deemed  unsatisfactory  was  flagged  and  not  processed  further. 

For  location  data,  the  RTK  GPS  receivers  present  a  Fix  Quality  value  that  relates  to  the  quality  / 
precision  of  the  reported  position.  A  Fix  Quality  (FQ)  value  of  3  (RTK  Fixed)  is  the  best 
accuracy  (typically  3-5  cm  or  better).  A  FQ  value  of  2  (RTK  Float)  indicates  that  the  highest 
level  of  RTK  has  not  be  reached  yet  and  location  accuracy  can  be  degraded  to  as  poor  as  -1  m. 
FQs  1  &  4  correspond  to  the  Autonomous  and  DGPS  operational  modes,  respectively.  Data 
collected  under  FQ  3  and  FQ  2  (at  the  discretion  of  the  data  analyst)  were  retained.  Any  other 
data  were  deemed  unsatisfactory,  flagged,  and  not  processed  further.  The  section  of  data 
containing  the  flagged  data  was  logged  for  future  re-acquisition  as  required.  Additionally,  any 
identified  gaps  in  data  coverage  were  brought  to  the  survey  team’s  attention  for  further  data 
collection.  Data  which  meet  these  standards  are  of  the  quality  typical  of  the  MTADS  system. 

Since  the  Discrimination  Array  operates  in  a  cued  mode,  the  data  QC  procedures  and  checks 
differ  from  the  survey  mode  instruments.  The  status  of  the  RTK  GPS  system  can  be  visually 
determined  by  the  operator  prior  to  starting  the  data  collection  cycle,  assuring  that  the  position 
and  orientation  information  are  valid  (FQ  3)  during  the  collection  period. 

Two  data  quality  checks  were  performed  on  the  TEM  array  data.  After  background  subtraction, 
contour  plots  of  the  signal  were  generated  for  the  25  transmit/receive  pairs  at  a  decay  time  of 
0.042  ms.  An  example  of  a  good  data  set  from  a  single  anomaly  with  a  large  SNR  is  shown  in 
Figure  5-9  for  Calibration  Area  item  16.  The  plots  were  visually  inspected  to  verify  that  there 
was  a  well  defined  anomaly  without  extraneous  signals  or  dropouts.  QC  on  the  transmit/receive 
cross  terms  was  based  on  the  dipole  inversion  results.  Our  experience  has  shown  that  data 
glitches  show  up  as  reduced  dipole  fit  coherence.  No  data  glitches  were  observed  during  our 
APG  demonstration,  so  data  collected  at  our  Blossom  Point  facility  for  a  105mm  HEAT 
Projectile  are  used  as  an  example.  The  fit  polarizabilities  as  a  function  of  decay  time  are  shown 
with  all  elements  (Figure  5-10)  and  with  element  21  excluded  (Figure  5-11),  with  fit  coherences 
of  0.699  and  0.992  respectively.  In  comparing  Figure  5-10  and  Figure  5-11,  the  expected  axial 
symmetry  of  the  minor  polarizabilities  is  restored  with  the  exclusion  of  element  21  in  this  case. 
The  problem  with  element  21  is  only  observable  at  late  time  and  therefore  would  not  be  captured 
in  the  monostatic  plot  review. 
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Figure  5-10  -  Polarizability  as  a  function  of  time  for  a  105mm  HEAT  Projectile  with  all  data 
included.  The  fit  coherence  for  all  elements  included  was  0.699. 
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Figure  5-1 1  -  Polarizability  as  a  function  of  time  for  a  105mm  HEAT  Projectile  with  element  21 
excluded.  The  fit  coherence  with  element  21  excluded  was  0.992. 

The  issue  of  when  multiple  objects  are  found  to  be  under  the  array  simultaneously,  generating 
overlapping  signatures,  can  also  be  addressed  at  this  point  in  the  data  QC  process.  An  example 
case  is  shown  in  Figure  5-12.  There  are  two  apparent  issues  in  the  data  set.  First,  there  appears 
to  be  a  small,  shallow  bit  of  scrap  on  top  of  the  target.  Second,  there  was  a  bit  of  scrap  present  in 
the  background  file  used.  This  latter  issue  is  seen  in  the  data  from  element  0.  Figure  5-13  shows 
the  fit  betas  obtained  using  all  of  the  data.  Figure  5-14  shows  the  fit  betas  obtained  excluding 
elements  0,5,10,11,15,16,17,20,21,22.  The  latter  betas  match  our  library  105mm  HEAT  betas 
very  well. 

Any  data  set  that  was  deemed  unsatisfactory  by  the  data  analyst,  was  flagged  and  not  processed 
further.  The  anomaly  corresponding  to  the  flagged  data  was  logged  for  future  re-acquisition. 
Data  which  meet  these  standards  are  of  the  quality  typical  of  the  MTADS  system. 
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Figure  5-12  -  Monostatic  QC  contour  plot  for  example  anomaly 


Figure  5-13  -  Polarizability  as  a  function  of  time  for  the  example  show  in  Figure  5-12  with  all 
data  included.  The  fit  coherence  for  all  elements  included  was  0.652. 
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Figure  5-14  -  Polarizability  as  a  function  of  time  for  the  example  show  in  Figure  5-12  with 
elements  0,5,10,11,15,16,17,20,21,22  excluded.  The  fit  coherence  for  the  remaining  elements 
was  0.985. 

5.5.4  Data  Handling 

Data  were  stored  electronically  as  collected  on  the  MTADS  vehicle  data  acquisition  computer 
hard  drives.  Approximately  every  two  survey  hours,  the  collected  data  were  copied  onto 
removable  media  and  transferred  to  the  data  analyst  for  QC/analysis.  The  data  were  moved  onto 
the  data  analyst’s  computer  and  the  media  recycled.  Raw  data  and  analysis  results  were  backed 
up  from  the  data  analyst’s  computer  to  optical  media  (CD-R  or  DVD-R)  or  external  hard  disks 
daily.  These  results  are  archived  on  internal  file  servers  at  NRL  or  SAIC.  Refer  to  Reference  7 
for  specific  details  on  the  magnetometer  system  file  formats.  Examples  of  the  Discrimination 
Array  file  formats  are  provided  in  Appendix  B.  All  field  notes  /  activity  logs  were  written  in  ink 
and  stored  in  archival  laboratory  notebooks.  These  notebooks  are  archived  at  NRL  or  SAIC. 
Relevant  sections  are  reproduced  in  demonstration  reports  such  as  this.  Dr.  Tom  Bell  is  the  POC 
for  obtaining  data  and  other  information.  His  contact  information  is  provided  in  Appendix  A  of 
this  report. 

5.5.5  Magnetometer  Survey  Data  Summary 

The  MTADS  tow  vehicle  and  magnetometer  array  were  mobilized  to  the  APG  Test  Site  on  May 

6,  2008.  The  magnetometer  portion  of  the  demonstration  was  conducted  the  following  day,  May 

7,  2008.  A  standard  MTADS  magnetometer  survey  of  the  Calibration,  Blind  Grid,  and  the 
Indirect  Fire  Area  of  the  Standardized  Site  was  conducted.  Anomaly  locations  were  manually 
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identified  from  the  data  for  the  Blind  Grid  and  in  a  combination  threshold  exceedance  /  manual 
review  method  for  the  Indirect  Fire  Area  as  described  in  Section  6.2  below.  A  data  segment 
around  each  anomaly  center  was  extracted  and  analyzed  using  the  UX- Analyze  subsystem  of  the 
Oasis  montaj  software  package  as  described  in  Section  6.2  to  fit  the  data  to  a  dipole  model  and 
extract  the  associated  fit  parameters  (position,  depth,  equivalent  size).  These  fit  results 
constituted  the  source  anomaly  lists  for  the  Blind  Grid  and  Indirect  Fire  Areas  for  the 
Discrimination  Array  stage  of  the  demonstration. 

In  practice,  the  majority  of  the  site  was  surveyed  as  a  whole  with  the  magnetometer  system  and 
the  resulting  data  were  analyzed  in  parallel  to  the  above  effort.  The  results  were  provided  to  the 
Program  Office  in  support  of  the  recent  reconfiguration  effort.  Anomalies  were  detected  in  a 
manner  similar  to  that  used  for  MTADS  data  sets  collected  as  part  of  the  ESTCP  UXO 
Discrimination  Study  at  the  former  Camp  Sibert  [7]. 

5.5.6  Calibration  Area 

The  ground  truth  for  the  Calibration  Area  is  known  and  the  collected  data  were  used  to  verify  the 
operational  status  of  our  systems  and  processing  procedures.  A  magnetometer  anomaly  map  of 
the  data  is  shown  in  Figure  5-15.  The  reported  location  and  identification  of  the  seeded  items  are 
also  indicated  in  the  figure.  An  amplitude  scale  of  ±10  nT  was  used  for  comparison  as  the 
threshold  exceedance  amplitude  for  the  Indirect  Fire  Area  was  set  at  4  nT. 
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Figure  5-15  -  Magnetometer  anomaly  map  for  the  APG  Calibration  Area. 

5.5.7  Blind  Grid 

Anomaly  locations  were  manually  identified  from  the  data  for  the  Blind  Grid.  These  results 
constituted  the  source  anomaly  lists  for  the  Blind  Grid  for  the  Discrimination  Array  stage  of  the 
demonstration.  A  coverage  map  for  the  magnetometer  system  is  shown  in  Figure  5-16.  The 
Blind  Grid  is  composed  of  400  cells  in  a  20x20  array  with  2m  on-center  spacing.  For  the 
magnetometer  stage  of  this  demonstration,  the  data  were  used  to  make  a  detection,  or  occupied  / 
not  occupied,  decision.  Two  hundred  and  forty-three  (243)  cells  were  identified  as  having  an 
anomaly  located  within  the  cell  boundary  and  classified  into  1  of  3  ranks.  Rank  1  anomalies 
(168  anomalies)  in  general  exhibited  a  good  response  and  were  added  to  the  Discrimination 
Array  anomaly  list.  The  magnetometer  response  over  the  Rank  2  anomalies  (46  anomalies)  were 
weak  and  may  have  been  caused  by  noise,  but  it  was  decided  to  add  them  to  the  Discrimination 
Array  anomaly  list  to  be  conservative.  The  Rank  3  anomalies  were  mainly  ones  with  a  lateral 
offset  of  greater  than  0.5m  from  the  center  of  the  cell.  Some  of  the  anomalies  were  also  fairly 
large.  The  Rank  3  anomalies  were  forwarded  to  ATC  for  post-reconfiguration  QC  purposes,  but 
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not  added  to  the  Discrimination  Array  anomaly  list.  An  Excel  spreadsheet  detailing  all  Blind 
Grid  anomalies  is  included  on  the  accompanying  DVD  (APG_BlindGrid_PickedCells.xls). 
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Figure  5-16  -  Magnetometer  system  coverage  map  for  the  APG  Blind  Grid. 
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5.5.8  Indirect  Fire  Area  Response  Curves 


The  Indirect  Fire  Area  (IDF)  is  seeded  with  three  munitions  types,  the  60mm  and  81mm  Mortar 
and  the  105mm  Projectile.  Prior  to  deployment  to  APG,  a  series  of  test-pit  measurements  were 
made  at  Blossom  Point  in  a  similar  manner  to  those  made  for  the  ESTCP  UXO  Discrimination 
Study  at  the  former  Camp  Sibert  [7].  Example  items  of  each  of  the  three  munitions  type  were 
acquired  from  ATC  as  part  of  a  standard  munitions  set  for  the  measurements.  Figure  5-17 
through  Figure  5-19  show  both  the  measured  peak  positive  response  for  each  item  and  the  dipole 
model  bounding  curves  for  each  item.  The  RMS  background  signal  level  for  several  ‘quiet’ 
patches  of  magnetometer  data  were  determined  and  averaged  to  get  a  measure  of  the  overall  site 
background  level.  The  average  value  was  found  to  be  1.5  nT  (la).  Based  on  the  system 
response  to  the  three  munitions,  the  minimum  threshold  exceedance  value  was  found  to  be  5.3 
nT  for  the  60mm  Mortar  at  a  depth  of  llx.  With  the  25%  safety  margin  applied,  a  final 
threshold  of  4  nT  was  set. 


Figure  5-17  -  Magnetometer  response  curve  for  the  ATC  standard  60mm  Mortar 
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5-18  -  Magnetometer  response  curve  for  the  ATC  standard  81mm  Mortar 


Figure  5-19  -  Magnetometer  response  curve  for  the  ATC  standard  105mm  Projectile 
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5.5.9  Indirect  Fire  Area 


The  Indirect  Fire  Area  is  a  1.3  ha  area  drawn  from  the  original  configuration’s  Open  Field.  The 
IDF  has  been  seeded  with  three  munitions  types,  the  60mm  and  81mm  Mortar  and  the  105mm 
Projectile.  Based  on  the  response  curve  results  presented  in  Section  5.5.8,  an  anomaly  detection 
threshold  of  4  nT  was  selected  and  used  for  an  initial  anomaly  detection  screen  of  the  IDF 
magnetometer  data.  Only  the  positive  lobe  of  the  dipolar  response  is  detected  in  this  fashion. 
1,301  anomalies  were  detected  in  the  data  set.  A  data  segment  around  each  anomaly  center  was 
extracted  and  analyzed  using  the  UX-Analyze  subsystem  of  the  Oasis  montaj  software  package 
as  described  in  Section  6.2  to  fit  the  data  to  a  dipole  model  and  extract  the  associated  fit 
parameters  (position,  depth,  equivalent  size).  These  results  were  then  given  to  an  experienced 
data  analyst  to  review  each  anomaly  individually,  manually  adjust  the  data  segment  boundary  as 
necessary,  and  review  the  data  set  as  a  whole  for  any  anomalies  missed  by  the  automated  routine. 
Fit  parameters  for  the  final  list  of  1,310  anomalies  were  determined  and  are  provided  as  an  Excel 
spreadsheet  on  the  accompanying  DVD  (APG_IDF_FitResults.xls).  The  data  analyst  classified 
each  anomaly  into  one  of  the  ten  categories  listed  in  Table  5-2.  The  694  anomalies  in  Ranks  1, 
2,  3,  and  7  were  added  to  the  IDF  anomaly  list  for  the  Discrimination  Array.  The  anomalies  in 
the  other  categories  could  reasonably  be  excluded  as  not  potentially  arising  from  one  of  the  three 
seeded  munitions. 

Table  5-2  -  Ranking  categories  for  IDF  magnetometer  anomalies  and  number  of  anomalies  in  each 
category 


Rank 

Number 

of 

Anomalies 

Description 

1 

423 

Decent  fit  with  a  size  between  a  60mm  and  105mm 

2 

65 

Poor  fits  (<.8  fit  coherence)  but  would  recommend  taking  cued  data  because  of  the 
shape/size  of  the  anomaly 

3 

132 

Would  not  fit  but  would  recommend  taking  cued  data  because  of  the  shape/size  of 
the  anomaly 

4 

334 

Fits  but  size  too  small  for  a  60mm  (A  rough  cutoff  of  0.030m  in  the  Fit  Size 
channel  was  used) 

5 

79 

Anomaly  caused  by  array  bounce  noise/geology 

6 

12 

Anomaly  caused  by  data  processing  artifacts 

7 

74 

Fits  but  show  high  remnant  magnetization  (A  conservative  cutoff  of  80  degrees  in 
the  Fit  SolidAngle  was  used) 

8 

101 

Poor  fit  (<.8  fit  coherence)  likely  small  clutter  or  noise 

9 

85 

Would  not  fit  -  likely  small  clutter  or  noise 

10 

5 

Fits  but  size  larger  than  a  105mm  (A  cutoff  of  0.175m  in  the  Fit  Size  channel  was 
used) 

A  magnetometer  system  coverage  map  is  shown  in  Figure  5-20.  The  two  white  areas  shown  in 
Figure  5-20  represent  two  portions  of  the  Indirect  Fire  Area  that  were  not  accessible  to  the 
magnetometer  system  during  the  first  portion  of  the  demonstration. 
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Figure  5-20  -  Magnetometer  system  coverage  map  for  the  APG  Indirect  Fire  Area. 

5.5.10  Discrimination  Survey  Data  Summary 

The  MTADS  tow  vehicle  and  Discrimination  Array  were  mobilized  to  the  APG  Test  Site  during 
the  week  of  June  9,  2008.  A  shipping  container  was  delivered  on  May  30,  2008  to  augment  the 
onsite  secure  storage  facilities.  The  Discrimination  Array  portion  of  the  demonstration 
commenced  on  June  16,  2008  with  a  data  collection  in  the  Calibration  Area.  Cued  surveys  were 
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conducted  of  the  Calibration,  Blind  Grid,  and  the  Indirect  Fire  Area.  Data  collection  was 
completed  on  June  23,  2008.  The  system  was  demobilized  later  that  week  to  our  Blossom  Point 
facilities.  The  shipping  container  was  removed  on  June  27,  2008. 

The  array  was  positioned  roughly  over  the  center  of  each  anomaly  on  the  source  anomaly  list  for 
each  area  and  a  data  set  collected.  Each  data  set  was  then  inverted  using  the  data  analysis 
methodology  discussed  in  Section  6.0,  estimated  target  parameters  determined,  and  ultimately  a 
classification  made  for  each  item  for  the  Blind  Grid  and  Indirect  Fire  Area. 

5.5.11  Calibration  Area 

The  ground  truth  for  the  Calibration  Area  is  known  and  the  collected  data  were  used  to  verify  the 
operational  status  of  our  systems  and  processing  procedures.  Each  of  the  66  seed  locations  in  the 
Calibration  Area  was  investigated  in  turn  and  the  resultant  fit  parameters  used  to  cross-check 
against  and  augment  our  library  of  responses  to  known  munitions  items.  Data  collection 
occurred  on  June  16,  2008.  The  background-corrected  data  set  for  each  anomaly  is  included  on 
the  accompanying  DVD. 

5.5.12  Blind  Grid 

The  Blind  Grid  is  composed  of  400  cells  in  a  20x20  array  with  2m  on-center  spacing.  The  Rank 
1  and  2  anomalies  (214  anomalies)  identified  in  the  magnetometer  survey  (See  Section  5.5.7) 
formed  the  Blind  Grid  anomaly  list  for  the  Discrimination  Array.  ATC  staff  placed  a  plastic  pin 
flag  on  the  center  of  each  of  the  214  anomalies  investigated  prior  to  this  portion  of  the 
demonstration.  Data  collection  occurred  during  the  period  June  16  -  17,  2008.  The  background- 
corrected  data  set  for  each  anomaly  is  included  on  the  accompanying  DVD. 

5.5.13  Indirect  Fire  Area 

The  Indirect  Fire  Area  is  a  1.3  ha  area  drawn  from  the  original  configuration’s  Open  Field.  The 
IDF  has  been  seeded  with  three  munitions  types,  the  60mm  and  81mm  Mortar  and  the  105mm 
Projectile.  The  694  anomalies  in  Ranks  1,  2,  3,  and  7  (See  Section  5.5.9)  were  added  to  the  IDF 
anomaly  list  for  the  Discrimination  Array.  Data  collection  occurred  during  the  period  June  17  - 
23,  2008.  The  background-corrected  data  set  for  each  anomaly  is  included  on  the  accompanying 
DVD. 

5.5.14  Indirect  Fire  Area  Survey  Patterns 

The  navigation  software  for  the  Discrimination  Array  provides  the  capacity  to  guide  the  operator 
with  cm-level  accuracy  to  place  the  array  over  the  center  of  any  anomaly  on  the  anomaly  list. 
Determining  the  most  efficient  method  with  which  to  order  the  anomalies  for  data  collection  was 
an  additional  part  of  this  demonstration.  The  IDF  was  broken  up  into  130m  long  and  20m  wide 
blocks  and  different  ordering  techniques  were  tested  and  evaluated.  The  three  techniques 
evaluated  were: 
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Placing  a  plastic  pin  flag  on  each  anomaly  location  and  allowing  the  operator  and 
spotter  personal  to  pick  the  optimum  path  on-the-fly.  The  flag  was  removed  from 
each  anomaly  after  data  collection. 


402740 _ 402760 _ 402780 _ 402800 _ 402820 
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Figure  5-21  -  A  portion  of  the  Indirect  Fire  Area  with  anomaly  locations  flagged. 
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Reordering  the  anomaly  list  into  a  pre-planned,  alternating  direction  path  where  the 
anomalies  were  binned  into  2m-wide  swaths.  Each  anomaly  was  investigated  in 
order  with  a  single  spotter  recording  completion  of  data  collection. 
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Figure  5-22  -  A  portion  of  the  Indirect  Fire  Area  with  anomalies  binned  into  2-m  swaths.  The 
anomalies  are  shown  in  black,  the  planed  route  is  shown  in  red. 
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c)  Reordering  the  anomaly  list  into  a  pre-planned,  alternating  direction  path  where  the 
anomalies  were  binned  into  Im-wide  swaths.  Each  anomaly  was  investigated  in 
order  with  a  single  spotter  recording  completion  of  data  collection. 
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Figure  5-23  -  A  portion  of  the  Indirect  Fire  Area  with  anomalies  binned  into  1-m  swaths.  The 
anomalies  are  shown  in  black,  the  planed  route  is  shown  in  red. 

Each  of  the  three  techniques  was  evaluated  for  an  entire  20-m  segment  of  the  field.  All  three 
techniques  were  found  to  be  equally  effective  for  our  practiced  survey  team,  especially  for  the 
vehicle  operator.  Production  rates  of  greater  than  200  anomalies  /  day  were  maintained  for  the 
entire  demonstration.  Based  on  discussions  with  the  operator,  the  2-m  swath  technique  requires 
significant  skill  in  precisely  placing  the  sensor  array  on  the  anomaly  including  while  backing  up. 
The  narrower  potential  solid  angle  for  the  1-m  swaths  is  expected  to  make  it  easier  for  less- 
skilled  operators  to  maintain  peak  efficiency.  The  pin  flag  technique  is  also  very  efficient  but 
requires  a  significant  amount  of  setup  and  takedown  for  the  flag  placement. 


42 


5.6  VALIDATION 


With  the  exception  of  the  Calibration  Grid,  the  ground  truth  for  the  Standardized  sites  is  held 
back  from  individual  technology  demonstrators  to  preserve  the  utility  of  the  Blind  Grid  and  Open 
Field  Areas.  Results  from  the  Blind  Grid  and  the  Indirect  Fire  Area  were  submitted  to  ATC  for 
performance  evaluation.  Scoring  results  have  been  received  and  are  available  [12].  A  summary 
of  the  results  are  given  in  Section  7.0. 

6.0  DATA  ANALYSIS  PLAN 

The  MTADS  Discrimination  Array  is  a  cued  UXO  classification  system.  There  were  two  parts 
to  the  data  collection  in  this  demonstration:  an  initial  magnetometer  survey  which  served 
primarily  to  locate  targets,  and  TEM  array  measurements  over  the  targets  detected  in  the 
magnetometer  survey.  The  TEM  data  were  used  for  target  classification  and  discrimination. 
Target  depths  determined  from  the  initial  magnetometer  survey  data  were  reviewed  but  not  used 
cooperatively  with  the  TEM  array  data  in  the  target  classification  and  discrimination  analysis  as 
discussed  in  Section  6.3. 

6.1  PREPROCESSING 

The  collected  raw  magnetometer  data  were  preprocessed  on  site  for  quality  assurance  purposes 
using  standard  MTADS  procedures  and  checks.  The  various  data  files  were  merged  and 
imported  into  a  single  Oasis  montaj  (v6.4,  Geosoft,  Inc.)  database  using  custom  scripts 
developed  from  the  original  MTADS  DAS  routines  which  have  been  extensively  validated. 

As  part  of  the  import  process  any  data  corresponding  to  a  magnetometer  outage,  a  GPS  outage, 
or  a  vehicle  stop  /  reverse,  were  defaulted  or  marked  to  not  be  further  processed.  Defaulted  data 
were  not  deleted  and  can  be  recovered  at  a  later  time  if  so  desired.  Any  long  wavelength  features 
such  as  the  diurnal  variation  of  the  earth’s  magnetic  field  and  large  scale  geology  were  filtered 
from  the  data  (demedianed). 

The  located,  demedianed  magnetometer  data  were  then  imported  into  UX- Analyze  for  individual 
anomaly  analysis.  The  data  surrounding  the  center  of  each  selected  anomaly  were  extracted  and 
submitted  to  the  physics-based  models  resident  in  UX-Analyze  to  determine  anomaly  size, 
position,  and  depth.  A  spreadsheet  containing  details  of  the  anomaly  location  and  fit  parameters 
is  provided  on  the  accompanying  DVD.  The  located  demedianed  magnetometer  data  were 
exported  into  an  ASCII  format  for  delivery  and  for  archival  purposes  and  are  provided  on  the 
accompanying  DVD. 

The  TEM  array  has  25  transmitters/receiver  pairs.  For  each  transmit  pulse,  the  response 
measured  at  all  of  the  receivers  was  recorded  simultaneously.  For  each  target,  a  25  x  25  x  N  data 
array  is  generated,  where  N  is  the  number  of  recorded  time  gates.  The  current  system 
configuration  bins  the  data  in  121  logarithmically  spaced  gates.  During  the  preprocessing  step, 
the  recorded  signals  are  normalized  by  the  transmitter  currents  to  account  for  any  transmitter 
variations.  To  account  for  time  delay  due  to  effects  of  the  receive  coil  and  electronics,  we 
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subtract  0.028  ms  from  the  nominal  gate  times  [13].  The  delay  was  previously  determined 
empirically  by  comparing  measured  responses  for  test  spheres  with  theory.  The  measured 
responses  include  distortions  due  to  transmitter  ringing  and  related  artifacts  out  to  about  40  ps. 
Consequently  we  only  included  the  responses  from  decay  times  beyond  40  ps  in  our  analyses. 
This  leaves  115  gates  spaced  logarithmically  between  0.040  ms  and  25  ms. 

The  background  response  was  subtracted  from  each  target  measurement  using  data  collected  in  a 
nearby  target-free  region,  or  background.  Background  locations  were  selected  from  quiet  areas 
observed  in  the  magnetometer  anomaly  map.  All  of  the  background  measurements  were  inter- 
compared  to  evaluate  background  variability  and  to  identify  outlier  measurements  potentially 
corresponded  to  measurements  over  non-ferrous  targets.  We  have  not  observed  significant 
background  variability  in  our  measurements  at  our  Blossom  Point  test  site,  and  were  able  to  use 
blank  ground  measurements  from  100  meters  away  for  background  subtraction  on  targets  in  the 
test  field. 

Geo-referencing  of  the  array  data  is  based  on  the  GPS  data,  which  gives  the  location  of  the  center 
of  the  array  and  the  orientation  of  the  array.  Sensor  locations  within  the  array  are  fixed  by  the 
array  geometry.  Dipole  inversion  of  the  array  data  (Section  6.3)  determines  target  location  in 
local  array-based  coordinates.  The  local  position  was  then  transformed  to  absolute  coordinates 
using  the  array  location  and  orientation  determined  from  the  corresponding  GPS  data. 

6.2  TARGET  SELECTION  FOR  DETECTION 

Targets  were  selected  from  the  magnetometer  survey  data  using  the  threshold  exceedance 
method  described  in  Reference  7  using  a  physics-based  threshold  in  Oasis  montaj  for  the  Indirect 
Fire  Area.  An  occupied  or  not  occupied  decision  was  had  for  each  Blind  Grid  cell  by  an 
experienced  data  analyst.  The  data  chips  associated  with  the  detected  anomalies  were  then 
processed  in  an  automatic  processing  mode  of  UX-Analyze.  The  results  of  the  automatic 
processing  were  then  reviewed  in  UX-Analyze ’s  interactive  mode  to  allow  operator  experience 
to  be  included  in  the  target  selection.  A  small  number  of  additional  anomalies  were  identified 
by  the  operator  and  added  to  the  anomaly  list.  The  anomaly  list  was  then  sorted  into  several 
categories  based  on  the  extracted  fit  parameters.  Some  of  these  categories  could  be  immediately 
excluded  from  further  study  as  the  munitions  of  interest  could  not  produce  such  a  response.  The 
collection  of  the  anomalies  in  the  remaining  categories  formed  the  final  detection  list  for  the 
Indirect  Fire  Area  and  served  as  the  target  list  for  the  Discrimination  Array  portion  of  the 
demonstration.  See  Section  5.5.5  for  further  details  on  the  generation  of  the  final  target  list. 

The  UX-Analyze  environment  for  anomaly  selection  and  analysis  contains  two  main  graphics 
tools.  One  displays  an  interpolated  amplitude  image  of  all  the  data  processed  at  the  site  for  the 
current  survey  and  allows  the  analyst  to  see  which  grids  of  the  site  have  been  surveyed  and  to 
select  grids  for  target  analysis.  The  area  selected  for  analysis  is  displayed  in  another  graphic 
window  in  which  the  analyst  visually  selects  targets  for  characterization.  In  both  of  these 
windows  there  are  options  for  changing  the  view  area,  the  mapping  color  palette  and  contrast,  the 
amount  of  information  overlaid  on  the  screen,  etc.  To  characterize  a  target  the  analyst  first 
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selects  the  magnetic  anomaly  by  drawing  a  boundary  around  the  anomaly  using  the  mouse. 
When  the  boundary  is  closed  the  data  inside  are  sent  to  a  dipole  fitting  algorithm  to  determine 
target  location,  depth  and  apparent  size. 

6.3  PARAMETER  ESTIMATION 

The  raw  signature  data  from  the  TEM  array  reflect  details  of  the  sensor/target  geometry  as  well 
as  inherent  EMI  response  characteristics  of  the  targets  themselves.  In  order  to  separate  out  the 
intrinsic  target  response  properties  from  sensor/target  geometry  effects  we  invert  the  signature 
data  to  estimate  principal  axis  magnetic  polarizabilities  for  the  targets.  The  TEM  data  are 
inverted  using  the  standard  induced  dipole  response  model  wherein  the  effect  of  eddy  currents 
set  up  in  the  target  by  the  primary  field  is  represented  by  a  set  of  three  orthogonal  magnetic 
dipoles  at  the  target  location  [14].  The  measured  signal  is  a  linear  function  of  the  induced  dipole 
moment  m,  which  can  be  expressed  in  terms  of  a  time  dependent  polarizability  tensor  B  as 

m  =  UBU^Ho 

where  U  is  the  transformation  matrix  between  the  physical  coordinate  directions  and  the 
principal  axes  of  the  target  and  Ho  is  the  primary  field  strength  at  the  target.  The  eigenvalues  Pi(t) 
of  the  polarizability  tensor  are  the  principal  axis  polarizabilities. 

Given  a  set  of  measurements  of  the  target  response  with  varying  geometries  or  "look  angles"  at 
the  target,  the  data  can  be  inverted  to  determine  the  (X,Y,Z)  location  of  the  target,  the  orientation 
of  its  principal  axes  ((t),0,v|/),  and  the  principal  axis  polarizabilities  (Pi,P2,P3)-  The  basic  idea  is  to 
search  out  the  set  of  nine  parameters  (X,Y,Z,(|),0,\[/,Pi,P2,P3)  that  minimizes  the  difference 
between  the  measured  responses  and  those  calculated  using  the  dipole  response  model. 

For  the  TEM  array  data,  inversion  is  accomplished  by  a  two-stage  method.  In  the  first  stage,  the 
target’s  (X,Y,Z)  dipole  location  is  solved  for  non-linearly.  At  each  iteration  within  this 
inversion,  the  nine  element  polarizability  tensor  (B)  is  solved  linearly.  We  require  that  this 
tensor  be  symmetric;  therefore,  only  six  elements  are  unique.  Initial  guesses  for  X  and  Y  are 
determined  by  a  signal-weighted  mean.  The  routine  normally  loops  over  a  number  of  initial 
guesses  in  Z,  keeping  the  result  giving  the  best  fit  as  measured  by  the  chi-squared  value.  The 
non-linear  inversion  is  done  simultaneously  over  all  time  gates,  such  that  the  dipole  (X,Y,Z) 
location  applies  to  all  decay  times.  At  each  time  gate,  the  eigenvalues  and  angles  are  extracted 
from  the  polarizability  tensor. 

In  the  second  stage,  six  parameters  are  used:  the  three  spatial  parameters  (X,Y,Z)  and  three 
angles  representing  the  yaw,  pitch,  and  roll  of  the  target  (Euler  angles  (|),0,x[/).  Here  the 
eigenvalues  of  the  polarizability  tensor  are  solved  for  linearly  within  the  6-parameter  non-linear 
inversion.  In  this  second  stage  both  the  target  location  and  its  orientation  are  required  to  remain 
constant  over  all  time  gates.  The  value  of  the  best  fit  X,  Y,  and  Z  from  the  first  stage,  and  the 
median  value  of  the  first-stage  angles  are  used  as  an  initial  guess  for  this  stage.  Additional  loops 
over  depth  and  angles  are  included  to  better  ensure  finding  the  global  minimum. 
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Figure  6-1  shows  an  example  of  the  principal  axis  polarizabilities  determined  from  TEM  array 
data.  The  target,  a  mortar  fragment,  is  a  slightly  bent  plate  about  ‘/2  cm  thick,  25  cm  long  and  15 
cm  wide.  The  red  curve  is  the  polarizability  when  the  primary  field  is  normal  to  the  surface  of 
the  plate,  while  the  green  and  blue  curves  correspond  to  cases  where  the  primary  field  is  aligned 
along  each  of  the  edges. 
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Figure  6-1  -  Principal  axis  polarizabilities  for  a  'A  cm  thick  by  25cm  long 
by  1 5cm  wide  mortar  fragment. 

Not  every  target  detected  during  the  magnetometer  survey  had  a  strong  enough  TEM  response  to 
support  extraction  of  target  polarizabilities.  All  of  the  data  were  run  through  the  inversion 
routines,  and  the  results  were  manually  screened  to  identify  those  targets  that  could  not  be 
reliably  classified.  Several  criteria  were  used  in  this  process:  signal  strength  relative  to 
background,  dipole  fit  error  (difference  between  data  and  model  fit  to  data),  the  visual 
appearance  of  the  polarizability  curves.  We  investigated  constraining  the  depth  in  the  TEM  fits 
using  the  magnetometer-derived  depth.  However,  a  comparison  between  the  derived  depths 
from  both  systems  with  the  Calibration  Grid  ground  truth  showed  no  significant  difference 
between  the  two,  as  shown  in  Figure  6-2.  Further,  performing  constrained  inversions  on  a  subset 
of  our  data  showed  generally  negligible  change  from  the  non-constrained  results.  We  thus 
decided  not  to  constrain  the  TEM  depths. 

For  moderate- to-good  SNR,  our  derived  polarizabilities  can  be  directly  compared  with  those  of 
our  target  library,  as  shown  in  Figure  5-5  through  Figure  5-7.  In  these  cases,  a  metric  was 
computed  based  on  how  well  the  amplitude  and  ratio  of  the  target  polarizabilities  match  those  of 
library  objects.  The  metric  runs  from  0  (terrible  match)  to  1  (perfect  match).  However,  for 
targets  with  lower  SNR,  or  for  cases  where  a  visual  inspection  suggested  a  match,  but  the  match 
quality  was  below  our  cutoff,  target  classification  and  discrimination  were  done  using  library 
matching  procedures  similar  to  those  used  by  Sky  Research,  Lawrence  Berkeley  Lab  and 
ourselves  in  the  ESTCP  Discrimination  Study  Pilot  Program.  In  our  version,  we  compare  the 
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quality  of  an  unconstrained  dipole  inversion  of  the  TEM  array  data  to  the  quality  of  a  dipole  fit 
constrained  by  a  set  of  principal  axis  polarizabilities  drawn  from  a  signature  library.  Fit  quality 
is  the  squared  eorrelation  coeffieient  between  the  model  fit  and  the  data.  If  the  ratio  of  the 
constrained  fit  quality  to  the  uneonstrained  fit  quality  (p)  is  one,  then  the  library  item  is  as  good  a 
mateh  to  the  data  as  possible.  If  the  ratio  is  small,  then  the  library  item  is  a  poor  mateh. 


Actual  Depth  (cm) 

Figure  6-2  -  Accuracy  of  depth  estimates  from  TEM  array  (la  =  6.1  cm) 
compared  to  MTADS  magnetometer  array  (la  =  5.4  cm) 

6.4  TRAINING 

Prior  to  the  demonstration  we  collected  training  data  in  air  for  all  of  the  14  standard  APG 
ordnance  targets.  These  data  were  used  to  generate  the  fit  library  entries.  Many  of  the  targets 
are  composites  of  two  or  more  distinet  parts,  like  a  steel  body  combined  with  an  aluminum  tail 
assembly.  Depending  on  the  distance  between  the  sensors  and  the  target,  such  items  can  exhibit 
a  range  of  slightly  different  EMI  signatures  corresponding  to  exeitation  from  different  directions. 
We  included  measurements  with  the  target  oriented  nose  up,  towards  the  sensor  array,  nose 
down,  away  from  the  array,  flat  and  obliquely  in  the  fit  library. 

Our  experienee  at  our  Blossom  Point  test  site  has  been  that  polarizabilities  determined  from  in¬ 
air  measurements  are  indistinguishable  from  those  determined  from  measurements  taken  over 
buried  targets.  We  used  the  data  eollected  from  the  APG  Calibration  Area,  whieh  contains 
several  instanees  of  eaeh  target,  to  establish  that  this  held  true  at  APG. 
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We  have  assembled  a  fairly  extensive  polarizability  database  for  clutter  items  recovered  from 
several  different  sites.  This  library  was  used  as  training  data  for  establishing  UXO/clutter 
discrimination  boundaries  on  the  direct  match  metric  and  on  the  coherence  ratio  p. 

6.5  CLASSIFICATION 

As  noted  in  Section  6.3,  for  anomalies  with  moderate-to-good  SNR  and  no  serious  overlap 
issues,  classification  was  done  using  the  direct  match  metric,  wherein  the  unconstrained 
polarizabilities  are  compared  with  those  of  library  objects.  We  had  previously  assembled  a  target 
signature  library  of  the  ordnance  types  present  at  APG,  and  supplemented  it  where  necessary 
with  measurements  of  test  objects  performed  on-site.  Using  the  results  of  the  Calibration  Grid, 
and  even  considering  Blind  Grid  and  IDF  anomalies  where  the  visual  match  to  a  library  object  is 
so  good  that  we  have  high  confidence  in  the  identification,  we  determined  a  cutoff  value  for  our 
metric  of  0.56  (reduced  to  0.44  for  the  25  mm),  above  which  the  target  is  considered  ordnance, 
and  below  which  it  is  considered  clutter.  This  can  be  considered  a  first  pass  for  classification, 
and  was  applied  to  all  anomalies. 

A  number  of  targets  had  poorer  SNR,  but  were  still  amenable  to  analysis.  For  these  cases,  we 
used  the  coherence  ratio  p  in  conjunction  with  the  direct  match  metric  as  our  classifier.  Library 
constrained  inversions  were  performed  on  all  these  targets,  and  were  also  applied  to  a  control 
group  consisting  of  clean  ordnance  and  clutter  targets  from  the  Calibration  Grid.  For  each  library 
inversion,  we  determine  the  library  object  which  best  fits  the  current  target.  We  then  form  the 
value  p,  equal  to  the  ratio  of  the  coherence  of  this  fit  to  that  of  the  unconstrained  fit.  From 
analysis  of  the  control  group  results,  a  cutoff  p  of  0.9815  was  determined,  above  which  all 
control  group  targets  are  ordnance,  and  below  which  they  are  all  clutter.  This  cutoff  was  then 
applied  to  the  poorer  SNR  targets  such  that  any  target  which  had  a  value  of  p  equal  to  or  greater 
than  0.9815  was  classified  as  ordnance,  regardless  of  the  value  of  its  direct  match  metric. 

6.6  DATA  PRODUCT  SPECIFICATION 

The  demonstration  was  conducted  at  the  APG  Standardized  Test  Site.  We  used  the  standard 
reporting  templates  for  the  Blind  Grid  and  the  Open  Field  shown  below  in  Figure  6-3  and  Figure 
6-4.  The  metrics  in  Section  3.0  were  calculated  directly  from  the  Scoring  Report  provided  by  the 
Standardized  Test  Site  administrators.  The  Response  Stage  value  was  the  maximum  signal  level 
above  background  from  the  magnetometer  anomaly.  The  calculation  of  the  discrimination  stage 
was  more  involved.  For  those  targets  classified  as  ordnance,  the  direct  match  metric  was  used  as 
the  ranking.  Additionally,  targets  which  we  could  not  analyze  as  described  in  Section  6.3  were 
classified  as  ordnance,  but  given  the  lowest  confidence  rating,  and  noted  as  such  in  our 
submission.  For  targets  classified  as  clutter,  a  division  was  made  between  targets  that  had 
moderate-to-good  SNR  and  little  to  no  overlap  with  adjacent  targets,  and  those  with  either  low 
SNR  or  more  serious  overlap  issues.  For  the  former  class,  ranking  was  based  on  the  direct  match 
metric,  while  for  the  latter,  ranking  was  based  on  a  combination  of  the  direct  match  metric  and 
the  peak  signal.  Classification  and  Type  will  be  determined  from  the  direct  match  metric  or  the 
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coherence  ratio  p,  as  applicable.  Easting,  Northing,  Depth,  Azimuth  and  Dip  all  come  from  the 
dipole  inversion  results. 
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Figure  6-3  -  Reporting  Template  for  APG  Blind  Grid. 
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Figure  6-4  -  Reporting  Template  for  APG  Indirect  Fire  Area. 
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7.0  PERFORMANCE  ASSESSMENT 


The  TEMTADS  array  was  constructed  in  2007  and  field  tested  at  the  APG  Standardized  UXO 
Test  Site  in  June  2008  [12],  For  the  APG  demonstration,  a  magnetometer  data  set  collected  the 
previous  month  at  the  newly  reconfigured  test  site  was  used  for  anomaly  detection  in  the  manner 
described  in  the  Magnetometer  /  Mkll  Demonstration  Plan.  Approximately  200  cells  in  the 
Blind  Grid  and  700  anomalies  in  the  Indirect  Fire  Area  were  interrogated  with  the  TEMTADS, 
averaging  200  anomalies/day  on  the  Indirect  Fire  Area.  After  processing,  ranked  dig  lists  were 
generated  and  submitted  to  ATC  for  scoring. 

The  performance  objectives  for  this  demonstration  were  summarized  in  Table  3-1,  and  are 
repeated  here  as  Table  7-1  with  an  additional  column  indicating  whether  or  not  success  was 
achieved.  The  results  for  each  criterion  are  then  discussed  in  the  following  sections.  The  ATC 
scoring  report  for  the  demonstration.  Reference  12,  provides  the  source  material  for  evaluating 
several  of  the  performance  objectives. 
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Table  7-1  -  Performance  Results  for  This  Demonstration 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Success? 

Quantitative  Objectives 

Reduction  of 

False  Alarms 

Number  of  false 
alarms 
eliminated  at 
demonstrator 
operating  point. 

•  Prioritized  dig 
list 

•  Scoring  report 
from  APG 

Reduction  of 
false  alarms  by  > 
50%  with  95% 
correct 

identification  of 
munitions 

Yes 

Location 

Accuracy 

Average  error 
and  standard 
deviation  in  both 
axes  for 
interrogated 
items 

•  Estimated 
location  from 
analysis 

•  Scoring  report 
from  APG 

AN  and  AE  <  10 

cm 

aN  and  aE  <  15 

cm 

Yes 

Production  Rate 

Number  of 
targets 

interrogated  each 
day 

•  Log  of  field  work 

75  targets  per 
day 

Yes 

Analysis  Time 

Average  time 
required  for 
inversion  and 
classification 

•  Log  of  analysis 
work 

15  min  per  target 

Yes 

Qualitative  Objectives 

Ease  of  Use 

•  Feedback  from 
operator  on  ease 
of  use 

Operator  comes 
to  work  smiling 

Yes 

7.1  OBJECTIVE:  REDUCTION  OF  FALSE  ALARMS 

This  is  the  primary  measure  of  the  effectiveness  of  this  technology.  By  collecting  high-quality, 
precisely-located  data,  we  expect  to  be  able  to  discriminate  munitions  from  scrap  and  frag  with 
high  efficiency. 

7.1.1  Metric 

At  a  seeded  test  site  such  as  the  APG  standardized  test  site,  the  metric  for  false  alarm  elimination 
is  straightforward.  We  prepared  a  ranked  dig  list  for  the  targets  we  interrogated  with  a  dig/no¬ 
dig  threshold  indicated  and  ATC  personnel  use  their  automated  scoring  algorithms  to  assess  our 
results. 
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7.1.2  Data  Requirements 


The  identification  of  most  of  the  items  in  the  test  field  is  known  to  the  test  site  operators.  Our 
ranked  dig  list  is  the  input  for  this  standard  and  ATC’s  standard  scoring  is  the  output. 

7.1.3  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  more  than  50%  of  the  non-munitions  items  can  be 
labeled  as  no-dig  while  retaining  95%  of  the  munitions  items  on  the  dig  list. 

7.1.4  Results 

This  Objective  was  successfully  met.  The  TEMTADS  surveyed  anomalies  detected  by  the 
MTADS  magnetometer  system  in  the  Blind  Grid  and  Indirect  Fire  Areas.  For  the  Blind  Grid 
Test  Area,  the  discrimination  stage  results  are  summarized  in  Table  7-2  and  Table  7-3  (subsets  of 
Table  6a  of  Reference  12),  broken  out  by  munitions  type  and  emplacement  depth.  For  the 
Indirect  Fire  Test  Area,  the  discrimination  stage  results  are  summarized  in  Table  7-4  and  Table 
7-5  (subsets  of  Table  6c  of  Reference  12),  broken  out  by  munitions  type  and  emplacement  depth. 
The  Probability  of  Detection,  is  defined  as  the  number  of  discrimination-stage  detections  / 
number  of  emplaced  munitions  in  the  test  site.  The  Probability  of  False  Positive,  Pfp‘^'®‘^,  is 
defined  as  the  (number  of  discrimination  stage  false  positives)/(number  of  emplaced  clutter 
items). 


Table  7-2  -  TEMTADS  Blind  Grid  Test  Area  Results 


disc 

All  Types 

105-mm 

81/60mm 

37/25-inin 

Munitions 

Scores 

0.97 

0.93 

0.97 

1.00 

0  to  4D 

1.00 

1.00 

1.00 

1.00 

4D  to  8D 

1.00 

1.00 

1.00 

1.00 

8D  to  12D 

0.67 

0.67 

0.00 

1.00 

Table  7-3  -  TEMTADS  Blind  Grid  Test  Area  Results 


disc 

All 

Masses 

0  to  0.25 
kg 

>0.25  to  1 
kg 

>1  to  10 
kg 

All  Depths 

0.01 

0.02 

0.00 

0.00 

0  to  0.15m 

0.01 

0.02 

0.00 

0.00 

0.15  to 

0.3m 

0.00 

0.00 

0.00 

0.00 

0.3  to 

0.6m 

N/A 

N/A 

N/A 

N/A 
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Table  7-4  -  TEMTADS  Indirect  Fire  Test  Area  Pa**'®*^  Results 


disc 

All  Types 

105-mm 

81/60mm 

37/25-inin 

Munitions 

Scores 

0.92 

0.93 

0.93 

0.91 

By 

Density 

High 

0.88 

0.92 

0.91 

0.80 

Medium 

0.94 

0.97 

0.89 

0.97 

Low 

0.94 

0.90 

0.97 

0.94 

By  Depth 

0  to  4D 

0.96 

0.94 

0.97 

0.97 

4D  to  8D 

0.92 

0.94 

0.92 

0.86 

8D  to  12D 

0.72 

0.75 

0.78 

0.67 

Table  7-5  -  TEMTADS  Indirect  Fire  Test  Area  Results 


disc 

All 

Masses 

0  to  0.25 
kg 

>0.25  to  1 
kg 

>1  to  10 
kg 

All  Depths 

0.04 

0.03 

0.02 

0.11 

0  to  0.15m 

0.04 

0.04 

0.02 

0.13 

0. 1 5  to 

0.3m 

0.04 

0.00 

0.06 

0.06 

0.3  to 

0.6m 

0.08 

0.00 

0.00 

0.20 

Efficiency  (E)  and  false  positive  rejection  rate  (R^)  are  used  to  score  discrimination  performance 
ability  at  two  specific  operating  points  on  a  ROC  curve:  one  at  the  point  where  no  decrease  in  Pa 
is  incurred  and  the  other  at  the  operator-selected  threshold.  Efficiency  is  defined  as  the  fraction 
of  detected  ordnance  correctly  classified  as  ordnance  and  the  false  positive  rejection  rate  is 
defined  as  the  fraction  of  detected  clutter  correctly  classified  as  clutter.  The  results  for  the  Blind 
Grid  and  Indirect  Fire  Test  Areas  are  summarized  in  Table  7-6  and  Table  7-7,  from  Tables  7a 
and  7c  of  Reference  12. 

Table  7-6  -  TEMTADS  Blind  Grid  Test  Area  Efficiency  and  Rejection  Rates 


Efficiency 

(E) 

False 

Positive 

Rejection 

Rate 

At  Operating 

Point 

0.99 

0.99 

With  No  Loss  of 

Pd 

1.00 

0.95 
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Table  7-7  -  TEMTADS  Indirect  Fire  Test  Area  Efficiency  and  Rejection  Rates 


Efficiency 

(E) 

False 

Positive 

Rejection 

Rate 

At  Operating 

Point 

0.98 

0.92 

With  No  Loss  of 

Pd 

1.00 

0.58 

7.2  OBJECTIVE:  LOCATION  ACCURACY 

An  important  measure  of  how  efficiently  any  required  remediation  will  proceed  is  the  accuracy 
of  predicted  location  of  the  targets  marked  to  be  dug.  Large  location  errors  lead  to  confusion 
among  the  UXO  techs  assigned  to  the  remediation  costing  time  and  often  leading  to  removal  of  a 
small,  shallow  object  when  a  larger,  deeper  object  was  the  intended  target. 

7.2.1  Metric 

As  above,  the  metric  for  location  accuracy  is  straightforward  at  a  seeded  test  site  such  as  the 
APG  standardized  test  site.  We  provided  an  estimated  position  for  all  targets  we  interrogated 
and  ATC  personnel  used  their  automated  scoring  algorithms  to  assess  our  results. 

7.2.2  Data  Requirements 

The  location  of  most  of  the  items  in  the  test  field  is  known  to  the  test  site  operators.  Our  dig  list 
is  the  input  for  this  standard  and  ATC’s  standard  scoring  is  the  output. 

7.2.3  Success  Criteria 

The  objective  will  be  considered  to  be  met  if  the  average  position  error  was  less  than  10  cm  in 
both  dimensions  (low  bias)  and  the  standard  deviation  of  each  dimension  was  less  than  15  cm 
(accurate  location). 

7.2.4  Results 

This  Objective  was  successfully  met.  The  location  accuracy  of  fit  parameters  generated  from  the 
Discrimination  Array  data  are  given  in  Table  7-8  and  Table  7-9,  taken  from  Tables  9a  and  9c  of 
Reference  12.  Horizontal  errors  are  not  calculated  for  the  Blind  Grid. 
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Table  7-8  -  TEMTADS  Blind  Grid  Test  Area  Location  Error  and  Standard  Deviation 


Mean 

(m) 

Standard 

Deviation 

(m) 

Northing 

N/A 

N/A 

Easting 

N/A 

N/A 

Depth 

0.02 

0.04 

Table  7-9  -  TEMTADS  Indirect  Fire  Test  Area  Location  Error  and  Standard  Deviation 


Mean 

(m) 

standard 

Deviation 

(m) 

Northing 

0.01 

0.05 

Easting 

0.01 

0.05 

Depth 

0.00 

0.06 

7.3  OBJECTIVE:  PRODUCTION  RATE 

Even  if  the  performance  of  the  technology  on  the  two  metrics  above  is  satisfactory,  there  is  an 
economic  metric  to  consider.  There  is  a  known  cost  of  remediating  a  suspected  munitions  item. 
If  the  cost  to  interrogate  a  target  is  greater  than  this  cost,  the  technology  will  be  useful  only  at 
sites  with  special  conditions  or  target  values.  Note,  however,  that  in  its  ultimate  implementation 
this  technology  will  result  in  reacquisition,  cued  interrogation,  and  target  flagging  in  one  visit  to 
the  site. 

7.3.1  Metric 

The  number  of  targets  interrogated  per  day  is  the  metric  for  this  objective.  Combined  with  the 
daily  operating  cost  of  the  technology  this  gives  the  per-item  cost. 

7.3.2  Data  Requirements 

Survey  productivity  was  determined  from  a  review  of  the  ATC  demonstration  field  logs. 

7.3.3  Success  Criteria 

For  this  first  demonstration,  the  objective  will  be  considered  to  be  met  if  at  least  75  targets  were 
interrogated  each  survey  day. 

7.3.4  Results 

This  Objective  was  successfully  met.  Data  collection  times  are  taken  from  Table  5  of  Reference 
12.  For  the  purposes  of  this  discussion,  a  typical  work  day  is  assumed  to  be  7  hours  of  active 
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data  collection  and  1  hour  of  daily  setup  and  tear  down.  214  anomalies  were  investigated  on  the 
Blind  Grid  over  the  course  of  10  hours  and  3  minutes,  or  1.43  work  days,  or  on  average  149 
anomalies  /  work  day.  694  anomalies  were  investigated  in  the  Indirect  Fire  Area  over  the  course 
of  32  hours  and  30  minutes,  or  on  average  150  anomalies  /  work  day.  In  fact,  through  the 
generosity  of  the  ATC  staff,  we  were  able  to  work  for  ten  or  more  hours  on  several  days  and 
achieved  production  rates  of  >  220  anomalies  /  day  for  three  days  at  the  end  of  the  first  week  of 
the  demonstration. 

7.4  OBJECTIVE:  ANALYSIS  TIME 

The  ultimate  implementation  of  this  technology  will  involve  on-the-fly  analysis  and 
classification.  The  time  for  this  will  be  limited  to  the  driving  time  to  the  next  anomaly  on  the 
list.  We  tracked  the  near-real-time  analysis  time  in  this  demonstration. 

7.4.1  Metric 

The  time  required  for  inversion  and  classification  per  anomaly  is  the  metric  for  this  objective 

7.4.2  Data  Requirements 

Analysis  time  will  be  determined  from  a  review  of  the  data  analysis  logs. 

7.4.3  Success  Criteria 

For  this  first  demonstration,  the  objective  will  be  considered  to  be  met  if  the  average  inversion 
and  classification  time  was  less  than  15  min. 

7.4.4  Results 

This  Objective  was  successfully  met.  The  average  inversion  time  per  target  was  approximately 

2.5  minutes  on  our  field  laptop  computer.  Including  this,  the  average  analysis  time  amounted  to 

12.5  minutes  per  anomaly.  Since  this  was  the  first  extensive  test  of  the  system  in  field  mode,  we 
took  the  opportunity  to  consider  various  discrimination  and  classification  methods,  some  of 
which  proved  unfruitful.  As  a  result  of  lessons  learned  from  this  undertaking,  we  expect  the 
average  analysis  time  for  future  field  runs  to  be  less  than  that  obtained  here. 

7.5  OBJECTIVE:  EASE  OF  USE 

This  qualitative  objective  is  intended  as  a  measure  of  the  long-term  usability  of  the  technology. 
If  the  operator  does  not  report  that  the  technology  is  easy  to  use,  shortcuts  that  can  compromise 
the  efficiency  of  the  technology  will  begin  to  creep  into  daily  operations. 

7.5.1  Data  Requirements 

This  objective  was  evaluated  based  on  operator  feedback. 
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7.5.2  Results 


This  Objective  was  successfully  met.  Based  on  vehicle  operator  feedback,  there  were  no 
significant  limitations  to  the  efficient  use  of  the  system  in  the  field.  Several  suggestions  were 
made  for  additional  improvements  to  the  navigation  and  data  collection  software.  They  have 
been  subsequently  incorporated.  Based  on  the  discussion  in  5.5.14,  anomalies  for  future 
demonstrations  will  be  ordered  along  Im  swaths  for  maximum  data  collection  efficiency. 

8.0  SCHEDULE  OF  ACTIVITIES 

Figure  8-1  gives  the  overall  schedule  for  the  demonstration  including  deliverables. 


Activity  Name 

2008 

Mar 

Apr 

May 

Jun 

Jul 

Aug 

1 

APG  Demonstration 

2 

Draft  Demonstration  Plan 

O 

3 

Final  Demonstration  Plan 

4 

Magnetometer  Data  Collection 

1 

5 

Discrimination  Data  Collection 

6 

Data  Analysis 

1 

1 

1 

7 

Ranked  Dig  Lists  to  ATC 

O 

March 

April 

May 

June 

July 

August 

Figure  8-1  -  Schedule  for  all  demonstration  activities  including  deliverables. 
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9.0  MANAGEMENT  AND  STAFFING 


The  responsibilities  for  this  demonstration  are  outlined  in  Figure  9-1.  Tom  Bell  and  Dan 
Steinhurst  are  the  co-PIs  of  this  project  and  jointly  fdled  the  roles  of  Site  /  Project  Supervisor. 
Glenn  Harbaugh  was  the  Site  Safety  Officer  and  Data  Acquisition  Operator.  His  duties  included 
data  collection  and  safety  oversight  for  the  entire  team.  Dan  Steinhurst  assisted  with  data 
collection  and  served  as  the  Data  Analyst  for  the  magnetometer  survey.  Tom  Bell  served  as  the 
Quality  Assurance  Officer  for  this  demonstration  and  oversaw  the  Data  Analysts.  Data  analysis 
support  for  the  Discrimination  Array  data  collection  effort  was  provided  by  Jim  Kingdon. 


Figure  9-1  -  Demonstration  management  and  staffing  diagram. 
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Appendix  A:  Points  of  Contact 


POINT  OF 
CONTACT 

ORGANIZATION 

Phone 

Fax 

e-mail 

Role  in 
Project 

Dr.  Jeff 
Marqusee 

ESTCP  Program  Office 

901  North  Stuart  Street, 

Suite  303 

Arlington,  VA  22203 

703-696-2120  (V) 
703-696-21 14(F) 
jeffrey.marqusee@osd.mil 

Director, 

ESTCP 

Dr.  Anne 
Andrews 

ESTCP  Program  Office 

901  North  Stuart  Street, 

Suite  303 

Arlington,  VA  22203 

703-696-3826  (V) 
703-696-21 14(F) 
anne.andrews@osd.mil 

Deputy 

Director, 

ESTCP 

Dr.  Herb 

Nelson 

ESTCP  Program  Office 

901  North  Stuart  Street, 

Suite  303 

Arlington,  VA  22203 

703-696-3726  (V) 
703-696-21 14(F) 
202-215-4844  (C) 
herbert.nelson@osd.mil 

Program 

Manger, 

MM 

Ms.  Katherine 
Kaye 

HydroGeoLogic,  Inc. 

11107  Sunset  Hills  Road, 
Suite  400 

Reston,  VA  20190 

410-884-4447  (V) 
kkaye@hgl.com 

Program 

Manager 

Assistant, 

MM 

Dr.  Dan 
Steinhurst 

Naval  Research  Lab 
Chemistry  Division 

Code  6110 

Washington,  DC  20375 

202-767-3556  (V) 
202-404-8119  (F) 
703-850-5217  (C) 
dan.  steinhurst@nrl.navy.mil 

Co-PI  and 
Data 

Analyst 

Mr.  Glenn 
Harbaugh 

Army  Research  Lab 

Building  512 

Blossom  Point  Rd. 

Welcome,  MD  20693 

804-761-5904  (V) 
glenn.harbaugh@nrl.navy.mil 

Site  Safety 
Officer 

Dr.  Tom  Bell 

SAIC 

200  12th  Street  South 
Arlington,  VA  22202 

703-414-3904  (V) 
703-413-0505  (F) 
301-712-7021  (C) 
thomas.h.bell@saic.com 

Co-PI  and 
Quality 
Control 
Officer 

Dr.  Jim 

Kingdon 

SAIC 

200  12th  Street  South 
Arlington,  VA  22202 

703-414-3872  (V) 
703-413-0505  (F) 
james.b.kingdon@saic.com 

Disc. 

Array  Data 
Analyst 

Mr.  Rick  Fling 

Aberdeen  Test  Center 

410-278-2999  (V) 
301-992-9080  (C) 
rick.fling@us.army.mil 

Test  Site 
Manager 

A-1 
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Appendix  B:  Data  Formats 


The  MTADS  Discrimination  Array  is  a  new  sensor  system  and  the  file  formats  have  not  been 
discussed  elsewhere.  The  contents  of  the  two  data  files  collected  for  each  anomaly  and  a  brief 
explanation  of  each  entry  are  provided  below. 

Position  (.GPS)  Files 

Antenna, X_Of f set , Y_Of f set , Z_Of f set , Easting/ Yaw, Northing/ Pitch, HAE/ Range 
Main, 0.000, 1.365,07730,316256.990,4254211.094,-25.934 
AVRl, -0 .778, -1.418, 0.740, 3. 40349, 0.007 61, 2. 8 82 
AVR2, 0.7 7 8, -1.4 18, 0.7 4 5, 1.557 18, 0.0 042 5, 1.554 

These  data  files  are  ASCII  format,  comma-delimited  files.  A  header  line  is  provided. 

Line  1  -  Header  information 
Line  2  -  Main  GPS  antenna  data 


Main 

0.000 

1.365 

0.730 

316256.990 
4254211 .094 
-25.934 


Antenna  Identifier 

Cross-track  distance  from  array  center 
Down-track  distance  from  array  center 
Vertical  distance  from  array  center 
Easting  (UTM,  m)  position  of  Main  antenna 
Northing  (UTM,  m)  position  of  Main  antenna 
Height-above-ellipsoid  (m)  position  of  Main  antenna 


Line  3  &  4  -  AVR  GPS  antenna  data  (AVRl  as  example) 


AVRl 
-0.778 
-1 .418 
0.740 
3.40349 
0.00761 
2 . 882 


-  Antenna  Identifier 

-  Cross-track  distance  from  array  center 

-  Down-track  distance  from  array  center 

-  Vertical  distance  from  array  center 

-  Yaw  of  AVR  vector  (radians.  True  North  referenced) 

-  Pitch  of  AVR  vector  (radians) 

-  Range  of  AVR  vector  (m) 


Sensor  (.TEM)  Files 

These  data  files  are  a  binary  format  generated  by  a  custom  .NET  serialization  routine.  They  are 
converted  to  an  ASCII,  comma-delimited  format  in  batches  as  required.  Each  file  contains  25 
data  points,  corresponding  to  each  Tx  cycle.  Each  data  point  contains  the  Tx  transient  and  the 
corresponding  25  Rx  transients  as  a  function  of  time.  A  pair  of  header  lines  is  also  provided  for, 
one  overall  file  header  and  one  header  per  data  point  with  the  data  acquisition  parameters.  A 
partial  example  is  provided  below. 
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Line  1  -  File  Header 


CPUms, PtNo, LineNo, Belt, BlockT, nRepeats, DtyCyc, nStk, AcqMode, GateWid, GateHOf f , T 
xSeq, GateT, TxI_Z, RxOZ_TxZ, RxlZ_TxZ, Rx2Z_TxZ, Rx3Z_TxZ, Rx4Z_TxZ, Rx5Z_TxZ, Rx6Z_T 
xZ,Rx7Z_TxZ,Rx8Z_TxZ,Rx9Z_TxZ,Rxl0Z_TxZ,RxllZ_TxZ,Rxl2Z_TxZ,Rxl3Z_TxZ,Rxl4Z_T 
xZ,Rxl5Z_TxZ,Rxl6Z_TxZ,Rxl7Z_TxZ,Rxl8Z_TxZ,Rxl9Z_TxZ,Rx20Z_TxZ,Rx21Z_TxZ,Rx22 
Z_TxZ ,  Rx2 3 Z_TxZ , Rx2  4  Z_TxZ , 

Line  2  -  Data  Point  Header 

0, 1, 0, 2E-06, 0.9, 9, 0.5, 3, 2,0. 05, 5E-05, 22, 


0 

1 

0 

2E-06 

0.9 

9 

0.5 

3 

2 

0.05 

5E-05 

22 


Start  time  in  ms  on  CPU  clock  (always  0) 

Data  Point  Number  (always  1) 

Line  Number  (always  0) 

Time  step  for  transients  (seconds) 

Base  period  length  (seconds) 

Number  of  Tx  cycles  in  a  base  period 
Duty  cycle 

Number  of  base  periods  averaged  (or  stacked) 
Data  Acquisition  Mode  (binned) 

Gate  width  as  fraction  of  its  own  time 
Hold-off  time  (seconds)  for  first  data  point 
Tx  ID  number  (sensor  number  +  10) 


Line  3  -  First  Data  Line  in  First  Data  Point 


r  r  r  r  r  r  r  r  r  r  r  r  05,2 

1.79793904939509E-05, 

4.84779418501355E-05, 

0.000116516308079121, 

05,3.70976690069955E- 

0.000100251591870083, 

6.05368361143584E-06, 

05,5.7401043406257E-0 

06,3.27817636140336E- 


.01102465120852,-4.7 
1.39366551389817E-05 
4 . 05641650778409E-05 
-2.49295973312366E-0 
05,-0.00012760664920 
5.19149917311475E-05 
-0 . 00012567180802577 
5,-5.14479298585597E 
05,-1 . 1886747308274E 


1949940100108E-05,- 
,  -2 .554  7  0  612  8112  71E-05, - 
,  6.73185201421361E-06,- 
6,  4.21216420108736E- 
6  97  9, -0 . 00  05103  6  634  53  93333,  ■ 
,3.71239440686929E-05,- 
4,2 .44747669528873E- 
-05,-9.42595187481444E- 
05, -5 .57 0222 4 7 62 024 lE-05, 
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